The tested version passed the combined functional and practical assessment.
GraphSec 0.18.0 loaded without parser errors, passed its logic tests and produced HTML, JSON and CSV reports plus a SHA-256 manifest during a real Microsoft Graph run. All 53 control areas appeared in the report. Unavailable data was not counted as passed.
Final privacy-reduced app-only run.
Run GSE-20260819-125632-CB3E7BC8 used certificate-based app-only authentication. The private key remained on the assessment host and the HTML report omitted raw technical evidence.
| Metric | Result | Meaning |
|---|---|---|
| Overall score | 27/100 | security score × assessment coverage |
| Security score for evaluated controls | 58/100 | technically evaluable controls only |
| Assessment coverage | 47% | 25 of 53 technically evaluated |
| Status | 6 passed · 17 warnings · 2 failed | plus 19 manual, 7 skipped, 1 unlicensed and 1 not applicable |
| Internal errors | 0 | no control with status “Error” |
| Outputs | HTML · JSON · CSV · manifest | SHA-256 hashes matched |
The two failed controls were genuine findings in the test tenant: no Intune compliance or managed app-protection policies were found. They were not script failures.
Four evidence types instead of one blanket success claim.
PowerShell parser: 0 errors. Regression tests for scoring, Conditional Access, app permissions, SMTP AUTH and tenant boundaries passed.
The UX test confirmed 53 criteria, 53 risk descriptions, 53 risk ratings and 53 completion-evidence sections plus five management priorities.
Two emergency accounts, one service account and two ownerless apps produced the expected evidence changes. A negative control remained unchanged and cleanup was confirmed.
The audit queries Microsoft Graph with GET. It creates local report files but contains no tenant-changing Graph write request.
Which statements are supported.
| Claim | Verdict | Evidence |
|---|---|---|
| “53 control areas” | Confirmed | 53 definitions in the script and 53 cards in the tested report. |
| “Read-only” | Confirmed with scope | No Microsoft Graph write requests. Local report files are created. Setup and lab helper scripts are separate from the audit run. |
| “HTML, JSON, CSV and manifest” | Confirmed | All four outputs were generated and their manifest hashes matched. |
| “Fully automated M365 audit” | Not supported | The catalog contains automatic, partially automatic and manual controls. Licences, permissions and APIs limit coverage. |
| “Unavailable data is not passed” | Confirmed | Unlicensed, permission-missing, skipped and manual states do not receive a positive control score. |
| “Detects tenant changes” | Confirmed for four scenarios | IAM-08, IAM-12, APP-03 and the EXT-04 negative control were tested; this is not a blanket guarantee for every possible change. |
| “Customer-ready report” | Technically confirmed | Structure, search, filters, management/admin views, print view and privacy mode passed automated acceptance. Professional sign-off remains the assessor's responsibility. |
What the current evidence does not claim.
- Exchange Online and Teams were disabled in the final app-only run; seven dependent controls were correctly skipped.
- The Security Alerts endpoint was unavailable in the test tenant and was classified as unlicensed or unavailable.
- PSScriptAnalyzer was not part of this verification.
- The live run is not a complete end-to-end test of every supported operating system, licence and authentication combination.
- BSI and German legal mappings are orientation aids, not a compliance certificate.
The positioning is accurate when the limitations remain visible.
The core positioning as a locally executed, source-available, read-only assessment package for experienced Microsoft 365 administrators and MSPs is supported by the code and tests. The previously over-broad public control catalog has been synchronized with the actual modes and limitations in version 0.18.0.
Evidence base: GraphSec-Audit.ps1 0.18.0, automated logic and report tests, controlled tenant-change test dated 17 August 2026, and the real app-only run dated 19 August 2026. Last reviewed: 19 August 2026.
The public catalog lists each control's mode, objective, mapping and technical assessment boundary.