The advancement and widespread adoption of computing technology has yielded services that could help mitigate the climate crisis. However, the retirement of obsolete equipment, the consumption of rare earth materials, and the escalating energy demands associated with massive data processing and cloud infrastructures have raised new environmental dilemmas. Existing design and development methodologies primarily focus on fulfilling functional requirements and improving performance. In this article, we argue that these methodologies must be augmented with sustainability considerations encompassing energy efficiency, material usage, longevity, and upgradability. Solutions at different layers of the system stack, from the physical to the application layer, must be integrated. Moreover, there should be a strong focus on the transparency of sustainability metrics across the whole computing continuum. Building on fruitful discussions at the International Lorentz Workshop on Future Computing for Digital Infrastructures, we advocate novel approaches in the design, development, and operation of the computing continuum.