A recurring request in mobile security procurement is a single grade for an application: a letter, a percentage, something that fits in a slide. OWASP's mobile standards appear to supply exactly that, and they are frequently presented as if they did.
They do not, and the gap between what they measure and what people assume they measure is wide enough to produce the wrong decision.
Two documents, two jobs
MASVS — the Mobile Application Security Verification Standard — is a list of security requirements grouped into categories. It describes what a well-built application should do: encrypt what it stores, validate what it receives, resist tampering.
MASTG — the Mobile Application Security Testing Guide — is the procedure manual. For each requirement it sets out how to verify it: what to look for, with which tool, and what constitutes evidence.
MASVS states the requirement. MASTG states how compliance is demonstrated. An audit that cites MASVS without MASTG has stated a conclusion without the method that produced it.
The practical test of that distinction is reproducibility. A finding recorded as "fails MASVS-NETWORK" cannot be verified by a second party. A finding recorded as "fails MASVS-NETWORK: networkSecurityConfig permits cleartext to api.example.com, observed in the merged manifest of build 4.2.1, SHA-256 …" can be reproduced, disputed and confirmed fixed.
The eight categories
MASVS organises requirements into eight groups:
| Category | Concerns | Typical failure |
|---|---|---|
MASVS-STORAGE |
What the application writes to disk, and how | Session tokens in shared preferences |
MASVS-CRYPTO |
Algorithms, key handling, randomness | Hardcoded keys; a non-cryptographic RNG |
MASVS-AUTH |
Authentication and session management | Sessions that never expire server-side |
MASVS-NETWORK |
Transport security, certificate handling | Cleartext permitted; no pinning on a payments path |
MASVS-PLATFORM |
Interaction with the OS: IPC, WebViews, permissions | Exported components without a permission guard |
MASVS-CODE |
Input handling, dependency hygiene, build configuration | debuggable surviving into release |
MASVS-RESILIENCE |
Anti-tampering, anti-debugging, obfuscation | No integrity checks on a financial application |
MASVS-PRIVACY |
Data collection and disclosure | Trackers not declared in the privacy manifest |
Droidwatch maps each finding to its category and produces a per-category score from 0 to 100, plus a letter grade. Penalties are weighted by severity: a critical finding removes 40 points from its category, a high 25, a medium 10, a low 3. Informational findings do not deduct.
The arithmetic is worth walking through once, because it determines how a grade should be read. A category carrying one critical finding scores 60. A category carrying four medium findings also scores 60. The two are not equivalent problems: the first is a single defect that probably has a single fix, the second is a pattern. A grade compresses that difference away, which is why the category score belongs next to the finding list rather than instead of it.
Note the direction of the scale, because it inverts the other number in the same report. The MASVS score runs 0 to 100 where higher is better. The risk score runs 0 to 100 where higher is worse. Two numbers, same range, opposite meaning — mixing them up in a report is a common and expensive error.
The limitation that matters
Here is the sentence that belongs beside every MASVS grade, and which Droidwatch embeds in the scorecard data itself rather than only in the interface:
MASVS measures security controls, not intentions. A well-built malicious application can score highly.
This is not a caveat added for caution. It follows directly from what the standard asks. Consider a banking trojan that pins its certificates, encrypts its local database with a key held in the Android Keystore, obfuscates its own strings and detects rooted devices.
Against MASVS that application is exemplary. It implements transport security correctly, handles storage correctly, and satisfies resilience requirements better than most legitimate applications — because its author has a direct operational interest in resisting analysis.
Its purpose is to overlay a bank's login screen and forward the intercepted credentials. MASVS was never designed to notice.
The inverse error is equally available. A legitimate retail banking application that stores a session token in world-readable preferences and permits cleartext to a logging endpoint will produce a benign risk verdict, because nothing about it is hostile. It is also not safe to ship. Neither number is wrong; each answers a question the other was not asked.
Where the two numbers belong
The practical consequence is that a mobile security report needs both numbers, used for different questions.
The risk verdict answers: is this application hostile? It draws on behavioural signals — overlay capability combined with accessibility abuse, SMS interception, dropper patterns, command-and-control infrastructure — and on reputation and rule matches.
The MASVS grade answers: is this application well built? It is the right instrument for a first-party application heading into release, or for a third-party component entering a supply chain.
Asking the first question of a MASVS grade produces the trojan described above: an A-grade piece of malware. Asking the second question of a risk verdict produces a benign verdict on an application that should not pass review.
| Question | Instrument | Audience |
|---|---|---|
| Is this hostile? | Risk verdict and findings | SOC, incident response, fraud |
| Is this well built? | MASVS scorecard | Application security, release review |
| Can we depend on it? | Both, plus supply-chain findings | Procurement, third-party risk |
Absence of evidence
One further distinction changes how a scorecard should be read.
A category with no findings is not a category that passed. It may be a category that was never assessed — because the relevant analysis did not run, or the application does not exercise that surface, or the evidence needed sits behind a runtime the analysis did not have.
That last case is the common one, and it is structural rather than accidental. MASVS-RESILIENCE asks whether an application detects tampering and responds to it; establishing the response generally requires executing the application under a debugger. MASVS-AUTH asks about session handling, much of which is server-side behaviour a package cannot reveal. A static analysis of an .ipa — which is all that is available for iOS, across its seven modules — will leave more categories unassessed than an Android analysis that can be paired with instrumentation.
Droidwatch marks these categories sin_evaluar (not assessed) rather than scoring them, and reports how many of the eight were actually evaluated. A scorecard showing six of eight unassessed is a different document from one showing eight of eight passed, and collapsing the two is how "we ran a MASVS assessment" becomes an assurance nobody verified.
This is the same principle that governs the rest of the report: a forensic tool must distinguish we looked and found nothing from we did not look.
Using this in practice
For a third-party application under suspicion, lead with the verdict and the behavioural findings. Cite MASVS where a specific control failure supports the case — cleartext transmission of credentials, for instance, is both a MASVS-NETWORK failure and evidence of intent when combined with an exfiltration endpoint.
For a first-party application before release, lead with MASVS. Work through the categories that scored worst, and use the corresponding MASTG procedures to reproduce each finding before fixing it. A finding that cannot be reproduced is a finding nobody can confirm has been fixed.
For a supply-chain assessment, read both and treat the unassessed categories as open questions rather than as passes. This is the context in which the count of evaluated categories matters most, because a component is frequently accepted on the strength of a scorecard nobody read past the grade.
For a recurring release gate, the MASVS categories are the stable measure and the risk verdict is the anomaly detector. A first-party application's category scores should move slowly and predictably; a sudden drop in MASVS-CODE between two builds usually means a dependency changed. Running that comparison automatically is what a pipeline gate is for.
Both numbers appear in the same report, alongside the MITRE ATT&CK Mobile mapping and the finding-level evidence. The report walkthrough covers where each of them sits and how the rest of the document is organised.
To see both numbers on a real application, upload an APK. No payment method required.