4.8.3 Measuring the AMM Controls

Metrics are defined by NIST [38] as "a standard of measurement that defines the conditions and the rules for performing the measurement and for understanding the results of a measurement.", and keep a close relationship to the concept of accountability being developed by A4Cloud. From a technical viewpoint, metrics are widely used as an instrument for verifying/monitoring the compliance of non-functional requirements, such as those related to security, privacy, or accountability. Metrics are also a tool that facilitates the decision-making process, since they can be seen as an input of the management review process of an organisation [39]. In this context, accountability metrics become an important aspect of the proposed AMM, since they can be considered as a means for showing that proper mechanisms for privacy, security and information governance are in place and indeed support accountability. However, in order to fulfil with this vision, it is necessary to relate the accountability controls from the AMM to the accountability metrics developed by A4Cloud. This is possible thanks to the approach proposed by A4Cloud to develop meaningful accountability metrics (please refer to details in section 4.9).

The elicited accountability metrics cover approximately 32% (10) of the CSA CCM controls resulting of the mapping to attributes. The other 21 controls do not have any metric associated to them. Obtained results also show that out of 39 accountability metrics, only 14 different metrics (approx. 35%) were actually related to the mapped CSA CCM. The resulting 10 measurable CSA CCM, are associated with metrics that can be assessed either automatically (e.g., Metric 26 Mean time to revoke users) or through human intervention (e.g., Metric 38 Total expenses due to compensatory damages).

The fact of having non-measurable CSA CCM controls does not mean that these cannot be assessed at all; on the contrary, in these cases the traditional audit practice will prevail and the control(s) will be evaluated through provided evidence while applying self-assessments or third-party assessments. Next we elaborate about the usage of these metrics from the perspective of the CSA cloud reference architecture (CSA EA).