Rapid population growth and economic development can lead to agricultural expansion, but also to farmland abandonment. While farmland abandonment may not be a concern in some agricultural systems, farmland abandonment among smallholder farmers can lead to food insecurity. Much has been learned about the determinants of farmland abandonment in Europe, but very little in Sub-Saharan Africa. This study examines the determinants of farmland abandonment in the rapidly urbanizing metropolis of Shai-Osudoku District in Ghana. We selected potential factors that may influence the decision to abandon farmland. In total, we conducted 151 face-to-face interviews with farming households in 2021 using structured questionnaires. Descriptive statistics and multiple linear regression were used to examine the effect of farm-level and socioeconomic factors that could determine farmland abandonment. The results showed that farm size and distance to the farthest farmland were important determinants of farmland abandonment. Based on the findings, our recommendations are that policies should be implemented to reduce undesired farmland abandonment and its long-term negative impacts on food security. For this reason, improving infrastructure to increase smallholders’ access to land and disseminating modern technologies to improve incomes from the agricultural sector can be concrete measures to counteract unwanted farmland abandonment.