We’ve been using Kernaro Assist daily for six months. Not in a controlled pilot. Not in theory. In live practice, on real repositories, with real architecture teams. What follows is an honest account of what worked, what surprised us, what cost more than we expected, and what we’re still figuring out.
Modeling: description drafting was the real win
We expected big time savings on capture. The theory was solid: Kernaro Assist would draft element descriptions, saving architects from the tedium of writing them.
What we found was more specific. Description drafting is fast. Really fast. An architect provides context (the system’s purpose, its stakeholders, key constraints) and Kernaro Assist proposes a description in seconds. The architect edits or approves. The time savings are real and substantial.
But here’s what surprised us: the bigger win was consistency and naming validation. In traditionally-authored repositories, you end up with wildly inconsistent naming for the same concept. Some elements use “payment processing,” others use “payment gateway,” others use “payment engine.” Some descriptions are four sentences, others are four words. Application naming follows no consistent pattern.
Kernaro Assist made this visible in a way that manual review doesn’t. When the AI is proposing names and descriptions based on your MDG conventions, inconsistencies become obvious instantly. Teams using Assist report far more consistent repositories than they had before. This doesn’t sound like a big deal until you try to query a repository for “all payment systems” and discover you’re getting different answers depending on which naming convention you searched for.
The governance overhead was higher than we anticipated. We expected: architect approves description, it goes live. What we found: approving AI-generated relationships required more rigor than approving descriptions. The AI can propose that “System A integrates with System B,” but the relationship properties (cardinality, stereotype, whether it’s synchronous or asynchronous) are often wrong. The first wave of teams using Assist discovered they were approving incorrect relationship metadata without understanding the implications.
This drove us to recommend tightening MDG profiles hard before using Assist for relationship generation. If your MDG clearly defines what relationship types are valid, what cardinality rules apply, and what stereotypes are required, Kernaro Assist’s proposal accuracy improves significantly. The teams that spent time hardening their MDG first saw governance overhead drop substantially. The teams that tried to use Assist without clear MDG profiles spent weeks in review-reject cycles, then went back and tightened the MDG.
The lesson: MDG isn’t optional when you’re using AI-augmented modeling. It becomes the guardrail that makes AI safe to use at speed.
Analysis: the question cost dropped, so people started asking
We expected faster impact analysis. The theory was solid: Kernaro Assist can run impact analysis queries faster than a human analyst querying manually. Time savings were predicted at 30-40%.
What we found was bigger. Yes, impact analysis got faster. But the availability of fast analysis changed which questions people asked.
In traditional practice, analysts have a finite time budget. If you can run one impact analysis per day, you choose them carefully. You do the analysis for the biggest decisions. You skip the smaller ones. You don’t go back and re-analyze decisions that seemed reasonable six months ago but now look questionable.
When analysis cost drops to “a few seconds,” people started asking questions they wouldn’t have bothered with before. The product team asked “what’s the dependency chain if we decommission this API?” not because they were planning to, but out of curiosity. The operations team asked “do we actually use this middleware anymore?” because the query cost was negligible. The security team asked “what customer data flows through this system?” not because they suspected a problem, but because they could.
We measured about a 40% increase in analysis queries in the first three months. Most were low-stakes curiosity, but some uncovered genuine issues: systems being used in ways nobody documented, dependencies that should have been removed years ago, data sensitivities that nobody tracked.
The productivity gain isn’t just the time saved on individual analyzes. It’s the patterns you spot when analysis is cheap enough that people ask the questions they should be asking. You learn things you didn’t know you didn’t know.
Governance: completeness checking works, configuration took longer
We expected automation of completeness checking. The Broadcast Event agents in Kernaro Assist can be configured to identify missing elements, validate that required metadata is present, spot gaps in relationship coverage. The theory was solid: automate the tedious audit work.
What we found: Broadcast Event configuration was more complex than the documentation suggested. The documentation makes it sound straightforward, define a rule, the agent runs it, you see exceptions. In practice, teams needed multiple iterations to define rules that actually caught the gaps they cared about without producing false positives.
One team spent two weeks getting their “application must have a documented owner” rule to work correctly. The rule needed to account for applications owned by external vendors, applications in legacy status, applications in planning stages. The initial rule triggered on all of these. They had to refine it multiple times. Another team configured a “business capability must be mapped to at least one strategic initiative” rule that was well-intentioned but eventually disabled because their strategic planning cadence meant it was out of sync with their capability model for three months per year.
Once the rules were configured correctly, they were reliable and valuable. Teams that invested the configuration time report that continuous completeness validation has replaced their annual audits. Exceptions are caught and flagged immediately, not discovered in a quarterly review.
The lesson: Broadcast Events are powerful, but they require upfront investment in understanding your rules well enough to encode them accurately. You can’t just decide the rule in the moment. You need to think through edge cases and validation logic before you implement.
Stakeholder engagement: fewer simple questions, better conversations
We expected fewer requests via email. Stakeholders would use Kernaro AI Hub to answer their own questions instead of pinging architects.
What we found was yes, but with a twist. The number of simple requests did drop. “Who owns system X?” now gets answered by Kernaro AI Hub. “What applications support process Y?” gets answered by a query. The volume of “ask an architect” work dropped by roughly 35-40%.
But the nature of the requests that still reach architects changed. The simple ones are gone. The requests that remain are more complex: architectural decisions with ambiguity, cross-system trade-offs, situations where the model gives you the answer but the stakeholder doesn’t trust it or doesn’t know how to interpret it.
The quality of the conversation improved. Instead of a 30-second “here’s who owns this system,” architects are having 30-minute conversations about whether retiring a system would create problems, how a business transformation would require systems to be reorganized, whether a new capability should be built or acquired.
This isn’t fewer conversations. It’s better conversations. The architects we work with consistently said this was the most valuable change: the elimination of rote context-gathering meant they could actually do architecture thinking instead of architecture service work.
What we’re still figuring out
Three things we don’t have figured out yet, and we’re honest about this.
Governance of AI-generated content at scale. When Kernaro Assist generates hundreds of descriptions and dozens of relationships per week, what does governance look like? We’ve moved from “every entry gets human review” to “exceptions get reviewed” to “spot checks happen regularly.” But we don’t have a mature model for balancing governance rigor with the ability to move at AI speed. This is still evolving.
Training new architects to review rather than author. Onboarding a new architect to a repository means teaching them what’s in it. Onboarding them to a repository that has Kernaro Assist means teaching them both what’s in it and how to review AI-generated content. The skills are different. We’re learning that “approval” requires different judgment than “creation.” We’re still refining the training for this.
Maintaining model accuracy as generation speed increases. If the repository is growing at 10x the speed because Kernaro Assist is drafting content, the model can drift into inconsistency faster. The governance mechanisms we’ve implemented catch most of this, but we’ve had instances where the model was technically consistent but practically confused, relationships were correct but emphasized the wrong aspects of the system architecture. We’re still working on detection and correction for this class of problem.
What this means for your practice
If you’re considering AI augmentation for your EA practice, here’s what six months of real-world use suggests:
Invest in MDG governance before you deploy Kernaro Assist. A tight metamodel makes AI-assisted modeling far more effective.
Be clear about what governance rules you actually want to enforce, and implement them in Broadcast Events. Don’t try to apply traditional human review processes to AI-generated content.
Plan for the productivity multiplier to reveal new work. When analysis becomes cheap, people will want more analysis. This is good. Plan for it.
Accept that the first quarter will be heavier on governance than expected. The payoff comes in quarters two and three, as the governance infrastructure matures and rules settle into place.
Rethink what you do with the time you recover. If modeling time drops by 40%, what will architects do with those hours? The best outcomes come from organizations that deliberately allocate recovered time to higher-value work: analysis, architecture thinking, strategic support.
Six months in, we believe in the value. The repositories we work with are more consistent, more complete, and faster to query. The architects using Assist are doing more analysis, better conversations, and less rote work. The governance overhead was real but manageable. And we’re learning something new every week about how to practice architecture at the speed of AI.
The Amplify offering walks your team through this entire journey. Not in theory. With architects who’ve lived it.
