The integration of trees on agricultural lands enhances carbon sequestration in biomass and soils while providing additional ecosystem services and livelihood benefits. However, agroforestry is not always accounted for in greenhouse gas (GHG) mitigation programs and international reporting requirements (e.g. UNFCCC) due to institutional and technical barriers. Recently, the 2019 Refinement to the IPCC guidelines provided guidance for eight types of agroforestry systems in tropical regions: Fallow, hedgerow, alley cropping, multistrata, parkland, shaded perennial, silvoarable and silvopasture. Yet, the technical capacity to determine the extent of those systems for reporting remains a challenge. On-farm trees cannot be detected at regional scales while relying on commonly available coarse spatial resolution satellite data. These analyses depend on estimates of the fractional cover of woody vegetation in a pixel or given area, which prevent metrics for the characterization agroforestry systems such as density and number of trees, location and crown sizes. Here we introduce a framework to detect, map and characterize agroforestry systems using high-resolution planet data and deep learning. We discuss preprocessing, training, prediction and post-processing of images for classifying and report accuracies for common Peruvian agroforestry systems. The TREES4CLIMA project will provide a step forward to improve agroforestry activity data and reduce uncertainties in national inventories.