Obesity is linked to numerous detrimental health outcomes, including type 2 diabetes (T2D), cardiovascular disease, liver disease and various cancers. The role of excess adipose tissue accumulation in specific regions in mediating these effects is poorly understood. For example, although there are established associations between visceral adipose tissue and negative health outcomes at the population level, the precise mechanisms linking specific adipose depots to systemic metabolic health remain unclear. This complexity is further compounded by the multifaceted interplay of genetics, behaviour, and environmental factors driving the accumulation of adipose tissue throughout the body. To discern the link between adipose tissue depots and their role in whole-body metabolism, we developed an AI approach to quantify adipose depots in a genetically diverse population of mice over time, within the same environment—something that would be highly challenging to achieve in human populations. By deeply phenotyping these mice, we uncovered links between specific adipose tissue depots and metabolic indices. Additionally, through genetic mapping analysis, we revealed potential genetic drivers of discrete adipose tissue depots over time. Specifically, we developed a convolutional neural network (CNN) combined with a regression output layer to predict actual adipose tissue weights from single-energy whole-body X-ray images. Applying this model to diverse outbred mice across three time points revealed positive associations between visceral adipose tissue and the Matsuda Index, fasting insulin, HOMA-IR, fasting glucose-to-insulin ratio, and QUICKI. In contrast, subcutaneous adipose tissue (SUBQ) showed fewer significant correlations. We then genetically mapped adipose tissue depot weight at these three time points and changes between them, identifying QTLs that may drive specific adipose tissue accumulation and subsequently poor systemic metabolic health. This work provides new insights into the genetic factors driving adipose tissue distribution and their specific roles in metabolic health, potentially paving the way for more targeted interventions in obesity-related conditions.