# Measurement tools for ecodesign¶

Authors and date

## Introduction¶

Measurement is one of the basis of any scientific approach to compare, analyze, quantify and monitor phenomena. In an ecodesign approach of digital services, measuring is essential to establish a state of the art, to compare practices or to validate the positive impact of the modifications made. It is also a required element to make life cycle assesment1.

However, it is necessary to rely on universal, reproducible, uniform reference units, as accurate as possible while being perennial in time. These are the main principles of metrology 2 , the science of measurement.

In terms of digital services ecodesign, there is currently no standardized unit of measurement. The multitude and complexity of possible architectures, as well as that of environmental impact indicators, make this standardization a major challenge. However, several initiatives to create eco-design assistance tools have emerged, in various forms, to support designers and developers in a virtuous approach.

We suggest that you take a look at some of these tools in order to understand what they really measure, their advantages but also their limits.

## EcoIndex¶

EcoIndex3 is a french website that offers to analyze a web page. The server hosting the EcoIndex site will browse the page in question, collect a number of metrics, and calculate a score (between 0 and 100) representative of the environmental performance of the web page. The higher the score, the more the site is considered eco-designed according to the calculation criteria described below. This service is free of charge, the objective put forward by its authors being above all that of making the general public aware of the environmental impacts of the web.

Please pay attention that at the time of writing, the site ecoindex.fr specifies that a new version is under construction and will probably be operational by the time you read this. If it's not the case, the following is still relevant to understand the GreenIT-Analysis plugin.

The calculation of this score is based on three main technical parameters:

1. the number of DOM4 elements, an indicator put forward by the authors as revealing the environmental impact of devices
2. the amount of data received by the browser (in KB), representing the environmental impact of network equipment
3. the number of HTTP requests made to load the page, corresponding to the number of accesses to servers in datacenters

Thus, the three aspects of a classical simplified architecture of a web service are taken into account: the end user's device, the network and the datacenters.

EcoIndex calculation formula

In order to take into consideration the disparity in the measurements of these three indicators, the data are divided into intervals containing the same number of data, known as quantiles. The measured value is then positioned in one of these quantiles (20 in this case) taking into account its proximity to the lower and upper bounds of the quantile.

Example of quantile calculation

The bounds of the 3rd quantile of the DOM are 75 (lower bound) and 159 (upper bound). If we analyze a web page containing 96 elements in its DOM, this number will be placed in this 3rdquantile since 96 is greater than 75 but smaller than 159. To take into account its proximity with the bounds of the quantile, we add to 3 the ratio $$\frac{(DOM - boundsInf)}{(boundsSup - boundsInf)}$$ , here thus $$\frac{(96 - 75)}{(159 -7 5)} = 0,25$$ . So in this example, $Quantil(DOM) =$3.25

These 3 values are then weighted to represent the disparity of the impacts of each of these aspects on the environment, with:

* a factor of <b>3</b> for the end user's device
* a factor of <b>2</b> for the data centers
* a factor of <b>1</b> for the network


$$EcoIndex =100 - \frac{5\times(3\times Quantil(DOM)+2\times Quantil(HTTP)+ Quantil(Ko))}{6}$$

Once this score has been obtained, it is ranked against a repository of other values collected on previously analyzed websites in order to assign it a rank. These are distributed on a scale from A to G. We thus obtain a comparative indication of the performance of the web page compared to the other tested pages.

However, the calculation of the EcoIndex does not exempt itself from a real expertise in eco-design. On the one hand, the formula itself is based on assumptions and estimations, related to current knowledges and technological maturity. This score cannot be considered as exact nor perennial in time. Indeed, the disparity of the impacts of the different aspects have no reason to remain the same, as well as the other values of the EcoIndex base which is intended to evolve along with the websites. This calculation does not take into account the type or complexity of operations performed in the datacenter, only the number of requests, which is not indicative of its real activity. Furthermore, it only takes into account the initial loading of the page. It is thus possible to strongly reduce the three values (complexity of the DOM, quantity of data exchanged and HTTP calls) by initially loading a minimalist page, then by allowing the update of page fragments according to user actions. These practices are not, however, less impactful for the environment. Finally, the comparison between several websites only makes sense if they perform similar functions. The multitude of existing website types, their functionalities, their complexity and the data they implement can easily justify large differences in EcoIndex score between two sites that have no functional relationship.

Nevertheless, this score has several uses:

• as part of a project to improve the eco-design of a digital service, following the evolution of this score page by page allows to validate the progress of the project. For example, it is possible to integrate this calculation automatically into the software testing and validation chain, as a monitoring indicator.
• when comparing two digital services providing the same functionalities (for example an article page), it allows to make a more virtuous choice.

But like any indicator, it can be misused if one takes advantage of the flaws in its calculation mode. Understanding the architecture and technologies behind the generation of the page remains essential in an eco-design approach.

Using this score to compare two web pages providing different services does not make much sense. The complexity of the DOM can for example be explained by the complexity of the business processes implemented on the page. The amount of data downloaded is intrinsically linked to its nature: a video for example will inevitably be heavier than text.

## GreenIT-Analysis¶

There is also the free extension GreenIT-Analysis 56 for Chrome and Firefox which performs this same calculation of the EcoIndex directly within your browser. It also evaluates the implementation of good ecodesign practices directly through the analysis of html and javascript code received by the browser.

This analysis is very useful for developers wishing to validate their approach, allowing them to analyze and identify areas for improvement.

Even if it is very simple to use, this tool still requires expertise in software design so that the conclusions drawn make sense. Some good practices may not be respected for very good reasons related to the technical context or the functionalities implemented in the web page.

## Carbonalyser¶

The Carbonalyser browser extension 7 is available for Firefox (and formerly Chrome), but also as a mobile application for Android. It was developed by members of the Shift Project. It records the amount of data that passes through the browser, and will apply a mathematical formula described in this 2018 Shift report8, it is the "1byte" model. This modelling, thought to take into account the energy consumption of the three thirds of digital services architecture (end users' devices, networks, datacenters), produces a direct estimate of the energy consumption related to web browsing. The choice of a geographical area allows to multiply this consumption by the corresponding energy mix [^mixenergetique] to obtain an estimate in C02 emission.

This indicator, which provides an analysis over a whole navigation time, is interesting for several aspects:

• it gives an order of magnitude to materialize the use of digital services
• it allows a quick comparison of sites integrating business processes that go beyond the simple web page
• it allows the comparison of physical and digital practices: for example, a videoconference versus a face-to-face meeting, or any other digitalization of activity

Nevertheless the only environmental impact highlighted here is that of CO2 emissions, via a CO2 equivalence calculation, based on consumption multiplied by the energy mix. However, the environmental impacts of information technologies are not limited to greenhouse gas emissions during the use phase. The depletion of natural resources, water consumption, soil pollution, acidification of aquatic environments, etc. are other issues associated with the entire life cycle of a service, from manufacturing to the end of life of all the devices needed. Moreover, nuclear power generation has a significant advantage in terms of CO2 emissions alone, without considering other environmental impacts. This makes the French energy mix largely favorable when compared to other locations that rely on fossil fuels (oil, fuel oil, gas). Given the globalization of the web, what sense does it make to consider that a website is less polluting because it is visited from France rather than from another country? Shouldn't our digital uses evolve in a systemic way?

Like the EcoIndex, this calculation is based on assumptions: for example, it relies on the average energy consumption of devices, in this case laptops and smartphones. It also considers that the user is connected to the network by a wifi connection without taking into account the significant differences between an ADSL and 4G or even 5G connection. It also does not take into account the complexity of the operations performed in the datacenter nor the real efficiency of these datacenters.

## Towards other measurement tools¶

These initiatives are essential for raising awareness and very useful for making simplified and quick comparisons of digital services. However, they are limited to those running in a browser, and are based on many assumptions and approximations.

In order to get an accurate and sustainable measurement it is necessary to dig deeper into the architecture of digital services. For example, the ideal way to monitor their energy consumption during the use phase would be to have power meters allowing the measurement of each hardware and software components. However, the increasingly distributed architecture of digital services makes the identification of the equipments used more and more complex.

Nevertheless, we are starting to see initiatives emerge such as Scaphandre 10, an open source software library allowing precise calculation of the energy consumption of a digital service in a datacenter. On the end user's side, very precise products are beginning to emerge such as Greenspector 11, which allows a fine analysis of the most energy-intensive software components in an application, as well as a precise estimation of their energy consumption associated with the device used. It is in this context that the Negaoctet [^negaoctet] project is also involved, working on the implementation of a complete database allowing the evaluation of the environmental impacts of digital services with a life cycle and multi-criteria approach. Unfortunately the data produced will not be fully public.

The production and sharing of homogeneous and measured data are however essential to allow, for example, the implementation of the environmental display of digital services necessary to implement the new french law aimed at informing consumers on the environmental impact of their data consumption.

## Uncharted territory¶

Measuring the impact of a digital service is a field in which most remains to be built, as much in the standardization of environmental impact indicators as in the tools allowing to measure the necessary values.

In addition to improve existing measuring devices, we must also double our efforts to collect and reference data on the manufacturing and end-of-life phases of equipment.It is needed in order to be able to integrate them more accurately into calculations related to the use of digital services.

It is also necessary to be able to take into consideration the environmental impacts linked to the design of software and its end of life. How much devices are used during the design phase? What are the team travel habits, what are the tools, technologies and software development practices? What will be done with the data and infrastructure needed to run the service at its end of life? What are the end-of-life triggers? These are all questions whose answers may reveal more or less significant environmental impacts, and which should be integrated into any measurement of the impacts of the digital service, in the same way as those of the manufacture and end of life of all the devices used during the use phase.

Unlike a traditional consumer good, the manufacturing phase of a software product does not stop at the beginning of its use phase. Functional, corrective, regulatory and security updates are frequent and often require the continued mobilization of a technical team during the entire use phase of the software. This is especially true for a web platform, which must necessarily evolve at least in line with changes in regulations, standards and the identification of security flaws. Thus, the environmental impacts of the design of a digital service is often constantly increasing, throughout its lifespan. However, no data or measurement standards currently exist on this subject.

Finally, the tools mentioned here focus mainly on energy measurements, often translated into CO2 equivalent. It is necessary to move towards the creation of tools offering precise measurements of other impacts, linked for example to the depletion of natural resources, the acidification of environments, the emission of fine particles or water consumption.

1. “Carbonalyser”: the browser extension which reveals the climate impact of internet navigation. The shift project. Available at theshiftproject.org

2. “Lean ICT: Towards digital sobriety”: Our new report on the environmental impact of ICT. The shift project. 2019. Available at theshiftproject.org [15/02/2022]