Automating product specification validation with AI: Our homemade tool CodeSpecChecker

From manual QA to automated consistency checking

Ensuring alignment between product specifications and code is one of the recurring challenges in software development. Documentation often evolves faster than implementations, manual reviews are repetitive and time-consuming, and inconsistencies can appear late in the release process, creating risks of functional gaps, rework, and delivery delays.

At Algoan, we constantly explore how technology can help us deliver more reliable, scalable solutions. During our most recent internal Hackathon around AI, aimed at improving internal workflows and experimenting with GenAI applications, one of the six projects led to the development of CodeSpecChecker, our first Large Language Model-powered tool dedicated to improving specification validation.

A new way to review PRs

Traditionally, checking whether a Pull Request aligns with its User Story has been a manual effort. Developers, QA engineers, and product managers have to compare code changes with documentation to make sure nothing is missing or inconsistent.

With CodeSpecChecker, this step becomes largely automated:

• A developer opens a PR in GitHub
  
• The tool retrieves the corresponding User Story from Notion
  
• An LLM analyzes the consistency between the code changes and the documented requirements
  
• The result is automatically posted as a comment in the PR.
  

This way, reviewers can immediately see where mismatches may occur, saving time and allowing them to focus on business logic and critical edge cases rather than mechanical checks.

Benefits for teams

In just a few weeks, CodeSpecChecker has already shown clear potential benefits:

• Higher code quality: discrepancies between documentation and code are detected earlier.
  
• Faster review cycles: repetitive checks are automated, freeing time for deeper analysis.
• Fewer specification drifts: improved alignment between documentation and code over time, reducing rework.

What’s next?

Like every prototype, CodeSpecChecker also opens new perspectives for improvement. Future development will focus on improving stability, supporting a wider range of specification formats, and delivering feedback that directly assists developers in resolving inconsistencies.

In summary

CodeSpecChecker started as a Hackathon idea and quickly became a working prototype integrated into our development workflows. By automating part of the validation process, it helps Algoan teams deliver software with greater speed and reliability.

For us, this project illustrates a broader approach: combining AI innovation with fintech expertise to not only strengthen our products, but also improve the way we build them.

Xiaoxiao Liu, Product Manager Intern at Algoan.