Planning How I’ll Evaluate My Research Project
Building on the module’s formative work, this post sets out how I’ll actually assess and evaluate my research once it’s underway, not just what I plan to study.
My research looks at whether AI is creating a strategic imbalance between cyberattack and cyber defence, and what that means for organisations. Before getting into the detail of my project plan, I wanted to work out something more basic first: how will I actually know if I’m right?
Deciding What Actually Counts as Evidence
I decided early on that I didn’t want to just gather a pile of interesting information and call it research. So I set myself a rule: I’ll only say AI is creating a real imbalance if I see attackers getting faster, wider-reaching, or cheaper results from a known attack pattern than defences can currently match, and I need to see this show up consistently across several cases, not just once.
Using Three Types of Evidence, Not One
Rather than relying on a single source, I’m combining three kinds of evidence and checking them against each other:
- Literature: to build the theory around how AI is changing attack speed, scale, and cost.
- Case Studies: from 2020 onwards, to see if that theory actually plays out in real incidents.
- Tool-Based Evidence: using MITRE ATT&CK Navigator and MITRE ATLAS, to see how attack stages are actually being carried out.
The literature and case material are analysed using thematic analysis, reading closely, coding recurring ideas, and grouping them into themes: offensive capability, defensive limitation, and organisational governance (Braun and Clarke, 2006).
If all three line up, I’ll feel more confident in a finding. If they don’t, I won’t smooth that over, I’ll report the disagreement itself as part of the result. This kind of evaluation approach doesn’t need to be locked in from day one either; it’s fine to build it step by step and adjust it as real evidence comes in (Kelly, Goodall and Lombardi, 2022).
The Tools I’m Using, and Where They Fall Short
I’m using four tools, but I want to be upfront about what each one can and can’t actually tell me.
VirusTotal
Checks files against over 70 detection engines, including behavioural analysis, so it can often catch AI-generated malware even when it’s just a variation of something already known.
MITRE ATT&CK Navigator
Maps out attacker tactics and stages. Because AI mostly speeds up known attack patterns rather than replacing them, this framework still applies, but on its own it can’t tell me what specifically was AI-driven.
MITRE ATLAS
Fills that gap by cataloguing AI-specific attack techniques, which I can then place onto ATT&CK’s stages.
Censys
Shows what’s publicly exposed online, helping test claims about AI speeding up reconnaissance. But exposure isn’t the same as an actual attack, so this only shows opportunity, not proof anything happened.
VirusTotal and MITRE ATLAS both only know what’s already been documented somewhere. Neither can catch a genuinely new AI technique that hasn’t been recorded yet. So my analysis will be strongest on established patterns and weakest at the cutting edge, and I think that’s an important limitation to be upfront about rather than pretend my tools can see everything.
Keeping the Project Itself on Track
Alongside evaluating my findings, I also need a way to notice early if a stage of the project is falling behind. So I’ve set some rough planning targets:
Literature Review
All three research themes covered.
Case Selection
~9–12 validated cases.
Tool Analysis
Each case run through the tools.
Synthesis
Conflicts worked through, not hidden.
These are estimates, not fixed dates, they’ll get sharpened once I write my actual project plan for Assignment 2. Structuring things around checkpoints like this isn’t just a formality either, it follows fairly standard project control practice, where you track status stage by stage rather than only checking at the very end (Kerzner, 2009).
I also want to be realistic about how demanding cross-checking three very different types of evidence actually is. Literature versus case studies will be my main analysis. Tool-based findings will come in afterwards as an extra check on a smaller set of well-understood cases, not a full three-way comparison across everything, since trying to force that evenly across the whole project isn’t realistic in the time I have.
Why Bother With All This Structure
It would be easy to treat this kind of planning as box-ticking, but research actually backs it up.
Recent studies on monitoring and evaluation frameworks show a real, measurable link between structured evaluation and better project outcomes: clearer alignment with what you’re actually trying to find out, more reliable results, and stronger accountability. Given how fast AI in cybersecurity moves, that structure is what will actually keep my findings credible rather than just descriptive.
- Braun, V. and Clarke, V. (2006) ‘Using thematic analysis in psychology’, Qualitative Research in Psychology, 3(2), pp. 77–101. Available at: https://doi.org/10.1191/1478088706qp063oa (Accessed: 9 September 2026).
- Kelly, L.M., Goodall, J. and Lombardi, L. (2022) ‘Developing a monitoring and evaluation framework in a humanitarian non-profit organisation using agile methodology’, Disaster Prevention and Management: An International Journal. Available at: https://doi.org/10.1108/DPM-11-2021-0312 (Accessed: 10 September 2026).
- Kerzner, H. (2009) Project Management: A Systems Approach to Planning, Scheduling, and Controlling. 10th edn. Hoboken, NJ: John Wiley & Sons.
- Nguru, J.K., Kithinji, M. and Kirimi, D. (2025) ‘Influence of monitoring and evaluation frameworks on the performance of projects within the parliamentary service commission of Kenya’, The Strategic Journal of Business & Change Management, 12(3), pp. 433–445. Available at: https://www.strategicjournals.com/index.php/journal/article/view/3337 (Accessed: 12 September 2026).
- Ovcina, A. and Arslanagic-Kalajdzic, M. (2024) ‘The role of monitoring and evaluation and project implementation management system for non-profit project performance in developing countries’, South East European Journal of Economics and Business, 19(1), pp. 63–76. Available at: https://doi.org/10.2478/jeb-2024-0005 (Accessed: 13 September 2026).