The influence of the ancient landscape configuration and the land use legacies on forest cover change has been thoroughly investigated in temperate and tropical forests while it remains barely explored at high-latitudinal regions. This study explores the landscape drivers leading forest cover changes and their influence on current forest attributes in the world’s southernmost forests (Argentina). Nothofagus antarctica forest cover changes were determined by combining multitemporal photo-interpretation of aerial photographs (1961) and satellite images (2016–2019). We analyzed changes in land covers and landscape diversity indexes. We also explored the potential relationships among descriptors of forest structure, soil or understory characteristics, and the magnitude of forest increase. Forest cover increased in 55 sites (72% of study area), and decreased in 21 sites (28% of study area). Cleared forests and forest edges decreased by half from past to the present, while woody encroachment increased. This resulted in a diversification of the current landscape by an increase in the number of land cover categories and Shannon–Wiener index. The increase in forest cover was positively influenced by land cover diversity and latitude, i.e., major increases at southern latitudes. Forests with greater recovery presented more similar understory attributes than those with lower recovery, while in other forest structures and soil properties, different magnitudes of recovery occurred. Sites with a high proportion of forest recovery evidence a reduction of past land use legacies (e.g., forage plant species introduced for livestock) at a local scale. These results expand our knowledge about the current natural dynamic of high-latitude forests, although the precise anthropic pressure changes (e.g., livestock density) leading to this process are still unclear and should be further investigated. Management practices should consider these spatial differences in the response to recovery of sub-Antarctic forests, such as the management of densely regenerated areas by thinning, or the restoration of degraded forests, as well as the monitoring of woody encroachment in natural grasslands and peatlands. Although we document an overall gain in N. antarctica forests, the results could vary when incorporating other forest types (N. pumilio, N. betuloides) and geographic zones (e.g., Andean mountains), due to the complex dynamics of these coupled human-natural systems.