A Data-Driven Framework for DevOpsMetric Evaluation and DeveloperExperience Proxy Analysis

Sammanfattning

DevOps Research and Assessment (DORA) metrics are used to measure software deliv-ery performance through indicators for delivery speed and operational stability. The fouroriginal DORA metrics are Deployment Frequency, Lead Time for Changes, Change FailureRate (CFR), and Mean Time to Restore (MTTR). Although these metrics are widely used,they can be difficult to calculate reliably when data is missing, inconsistent or hard to linkacross tools.This thesis designs and evaluates a lightweight DevOps analytics framework for cal-culating and assessing the trustworthiness of DORA metrics from DevOps tool data. Theframework is tested through a Ladok-based case study using simulated Jira and Jenkins-styleevents. Since access to live production data was limited, the evaluation used a controlledsimulated dataset with normal cases and edge cases, such as missing issue keys, failedpipeline runs, and unresolved bugs.The main contribution is an evaluation framework for judging when calculated DORAmetrics can be trusted. The metrics were evaluated through data coverage, linkage, ro-bustness, definition sensitivity, stability, metric strength, and limited developer experienceproxy signals.The results show that all four DORA metrics could be calculated, but they were notequally well supported. Deployment Frequency was the strongest supported metric inthe metric strength assessment because it required only Jenkins deployment data, had nomissing value, and had a clear operational definition. Lead Time for Changes dependedmore on Jira–Jenkins linkage and workflow history. CFR and MTTR were also computable,but their interpretation depended on how failures and recovery were defined.The thesis concludes that DORA metrics can support software delivery analysis, but onlywhen the data, links, and definitions behind each metric are clear. The framework can alsoshow possible developer friction through proxy signals, but it does not measure developerexperience directly.