Version Control for AI Systems
Master the art of maintaining accurate, helpful documentation for rapidly changing AI systems with practical version control strategies and sustainable maintenance approaches.
Table of Contents
âWait, that feature doesnât exist anymore?â
Maria stared at her screen in confusion. The documentation she was following had clear instructions for using the sentiment analysis threshold setting, but the option was nowhere to be found in the interface.
After 45 frustrating minutes and a desperate Slack message to the AI team, she got the answer: âOh, that was removed three versions ago. The docs must be outdated.â
Sound familiar? Youâre not alone. In the fast-moving world of AI, documentation that was perfect yesterday can become misleadingâeven dangerousâtoday.
In this module, weâll explore how to solve the seemingly impossible puzzle of keeping documentation accurate when your AI systems evolve faster than most people change their passwords. Think of it as building your very own documentation time machine, where every version of your system has the documentation it deserves.
Why AI Documentation Breaks Faster Than Regular Software
Letâs start with a confession: maintaining AI documentation is harder than for traditional software. Much harder.
Why? Imagine trying to document a house that keeps rearranging its rooms overnight. Thatâs AI for you.
Traditional software changes when humans deliberately update the code. But AI systems can change their behavior when:
- New training data arrives (suddenly your image classifier thinks fire hydrants are dogs)
- The world itself changes (your perfect 2019 prediction model meets the 2020 pandemic)
- The model drifts over time (like how your music recommendations slowly went from cool indie bands to baby shark remixes)
- Different model versions run simultaneously (version A for premium customers, version B for everyone else)
- Experimental branches multiply (the research team has 17 different versions theyâre testing)
âDocumentation is like milk, not wineâit doesnât get better with age.â â Every user who followed outdated AI documentation
Your Version Control Survival Kit: Three Core Strategies
To tame this documentation chaos, you need three powerful strategies working together.
Strategy 1: Version Alignment â Keeping Docs and Systems in Step
Imagine if your GPS showed you directions for the city as it existed five years ago. Youâd end up driving into buildings! Thatâs what happens when documentation and systems fall out of sync.
How to do it right:
- Match documentation versions to system versions: When your AI system becomes v2.5, your docs should too
- Use semantic versioning: Major.Minor.Patch helps users understand the scale of changes
- Major (1.0 â 2.0): âThe sentiment analysis feature now works completely differentlyâ
- Minor (2.0 â 2.1): âWeâve added support for three new languagesâ
- Patch (2.1 â 2.1.1): âFixed a typo in the French sentiment analyzerâ
- Add clear version labels everywhere: Every page should show âDocumentation for SuperAI v2.5.1â
- Include âThis feature is new in version Xâ badges: Help users understand whatâs recently changed
Real talk: I once spent a day debugging why my code wasnât working, only to discover I was reading documentation for a version two major releases ahead of what I had installed. Donât let this happen to your users!
Strategy 2: Multiple Version Support â Because Time Travel Is Real
In the AI world, not everyone upgrades at the same time. Some teams might be using your two-year-old model because itâs embedded in critical systems, while others are on the bleeding edge.
How to do it right:
- Create a version selector: Let users switch between documentation versions with a simple dropdown
- Build version-specific URLs:
docs.yourproduct.com/v2/vs.docs.yourproduct.com/v3/ - Show compatibility matrices: Clear tables showing which features exist in which versions
- Provide migration guides: Step-by-step instructions for upgrading (or downgrading!)
- Flag deprecated features: âThis feature will be removed in v4.0. Use feature X instead.â
The version selector in action: TensorFlowâs documentation lets users easily switch between versions, with a bright warning when viewing docs for anything other than the latest release. This simple UI pattern has saved countless hours of developer confusion.
Strategy 3: Automation â Because Lifeâs Too Short to Update Docs Manually
If youâre updating documentation by hand every time your model changes, youâre both a hero and doing it wrong. Automation is the key to documentation that keeps pace with rapid development.
How to do it right:
- Generate API documentation from code: Extract parameters, return types, and examples directly from source
- Pull model parameters automatically: âThe model accepts images of size 224Ă224â should never be typed manually
- Auto-create changelogs: Generate first drafts from commit messages and pull requests
- Build version previews: Automatically deploy documentation for unreleased versions
- Version test your docs: Automated checks that examples still work with the current version
Automation success story: One AI team I worked with reduced documentation errors by 80% by simply generating their parameter lists and API signatures directly from their Python code instead of manually updating them. The best part? It took just one day to set up.
Treating Documentation Like Code: The Secret to Sanity
Hereâs a revolutionary idea: what if we treated documentation with the same care and rigor as we treat code? Wild, I know!
Using Git: Not Just for Code Anymore
Git isnât just for tracking code changesâitâs also perfect for documentation:
- Create branches for documentation updates:
docs/add-sentiment-analysis - Make pull requests for documentation changes: Get reviews before publishing
- Write meaningful commit messages: âUpdated threshold parameterâ vs. âFixed docs to reflect the new default threshold of 0.75 for sentiment analysisâ
- See who changed what and when: Easily track down who updated a specific explanation
- Roll back problematic changes: Published something wrong? Revert the commit!
The magic button: Set up your docs system so non-technical team members can make small edits directly through a web interface, which creates Git commits behind the scenes. Best of both worlds!
Documentation Testing: Yes, Itâs a Thing
Would you deploy code without testing it? Probably not. So why would you publish documentation without checking it?
Tests to implement:
- Link checkers: Nothing frustrates users like clicking a link to nowhere
- Code example validators: Run the examples to make sure they still work
- Screenshot comparisons: Detect when UI images are outdated
- API call testers: Verify that your API examples return the expected results
- Spelling and grammar checks: Because âthe modle predicts sentimentâ isnât inspiring confidence
Quick win: Even just running a simple link checker before publishing can catch 90% of the most frustrating documentation problems.
Documentation CI/CD: Automate All the Things
If your development team has a continuous integration pipeline, your documentation should be in it too:
- Automatically build docs on push: No more âI forgot to rebuild the docsâ
- Deploy previews for pull requests: âHereâs how the documentation will look with your changesâ
- Run linting and style checks: Enforce consistent terminology and formatting
- Test code examples: Make sure they still work with the current API
- One-click deployment: Get approved changes to users immediately
The dream setup: When a developer changes an API parameter, the documentation automatically updates, tests run to verify everything still works, and the new docs deploy to the exact right versionâall without a human having to remember to do it.
Building Documentation Thatâs Designed to Last
Some documentation seems to age like fine wine, while other docs become vinegar almost immediately. Whatâs the difference? Design choices that account for change from the beginning.
The DRY Principle: Donât Repeat Yourself (Or Youâll Regret It)
Every piece of duplicated information is a ticking time bomb waiting to become inconsistent:
- Single source of truth: Define each concept, parameter, or process exactly once
- Content reuse: Use includes or snippets for information that appears in multiple places
- Parameterized content: âThe default threshold is â instead of hardcoded values
- Generated reference docs: Let the code itself be the source of truth for API details
- Centralized glossaries: Define terms once and reference them everywhere
The DRY disaster story: I once found a product where the default timeout value was mentioned in 37 different places in the documentation. When it changed from 30 seconds to 60 seconds, they missed updating 12 of those places. Users were confused for months.
Isolating Volatile Content: Quarantine What Changes Often
Not all documentation content changes at the same rate. Separate the stable from the volatile:
- Conceptual foundations: These rarely change and can be relatively stable
- Implementation details: These change frequently and should be isolated
- Interface specifics: These change with UI updates and should be clearly versioned
- Code examples: These break most often and need special attention
Pro tip: Create a âFrequently Changing Featuresâ section in your documentation that you review with every release. This acknowledges reality while focusing your maintenance efforts.
Documentation Architecture: Building to Last
Just like software architecture matters, how you structure your documentation determines how well it handles change:
- Modular structure: Independent sections that can be updated separately
- Clear ownership: Specific people responsible for specific sections
- Inheritance patterns: Let specific versions inherit from common content, overriding only what changed
- Update triggers: Documents that list what events should prompt documentation reviews
- Freshness metadata: âLast verified: March 15, 2023â on each page
The architecture win: A team I advised restructured their monolithic documentation into modules owned by different teams. Update frequency increased by 400% because everyone had a clear, manageable responsibility rather than an overwhelming collective one.
The Model Documentation Time Capsule: Special Considerations for AI
AI models have unique documentation needs that go beyond traditional software. Hereâs how to handle them:
Model Versioning: More Than Just a Number
For AI models, versions should capture:
- Training date: When the model was last trained
- Dataset version: What data it learned from (including date ranges)
- Architecture details: The structure and key parameters
- Performance benchmarks: Accuracy, precision, recall on test datasets
- Known limitations: What it struggles with or doesnât handle well
Real-world example: A facial recognition systemâs documentation included the specific demographic breakdown of its training data with each version, allowing users to understand potential bias areas as the training data evolved.
The Model Changelog: What Actually Changed
When releasing a new model version, document:
- Performance changes: âAccuracy improved from 87% to 92% on our benchmark datasetâ
- Behavioral differences: âThe model now prioritizes precision over recallâ
- Training methodology updates: âWeâve switched from supervised to semi-supervised learningâ
- Data changes: âAdded 50,000 new training examples from Asian marketsâ
- New capabilities: âNow supports sentiment analysis in 7 additional languagesâ
- Limitation improvements: âReduced gender bias in occupation predictions by 45%â
Why this matters: Users need to know not just that something changed, but how it might affect their specific use case. A general accuracy improvement might actually reduce performance for their particular scenario.
Model Lineage: The Family Tree
Document how your current model evolved:
- Derivation history: What previous models it builds upon
- Experiment records: What alternatives were tried
- Key breakthroughs: The innovations that made significant improvements
- Abandoned approaches: What didnât work (to prevent others from repeating mistakes)
- Future roadmap: What improvements are planned
Lineage in practice: Googleâs BERT model documentation clearly explains how it builds on previous transformer architectures, helping users understand its strengths and theoretical foundations.
Fighting Documentation Debt: Because It Compounds Like Credit Card Interest
Just like technical debt in code, documentation debt accumulates interest. The longer you leave it, the worse it gets.
Documentation Audits: Regular Health Checks
Schedule regular reviews to catch problems before users do:
- Accuracy verification: Is everything still technically correct?
- Coverage analysis: Are there undocumented features or parameters?
- Consistency checks: Does the documentation contradict itself?
- User path testing: Follow documentation instructions as if youâre a new user
- Feedback review: What are users struggling with most?
Audit approach: Set a calendar reminder for the first Monday of each month: âSpend 2 hours reviewing the most-viewed documentation pages for accuracy.â
Documentation Refactoring: Cleaning House
Sometimes you need to step back and improve the structure:
- Content reorganization: Improve how information flows
- Terminology standardization: Ensure consistent language
- Duplication elimination: Consolidate repeated information
- Legacy content archiving: Clearly separate old version documentation
- Navigation improvements: Make information easier to find
The refactoring catalyst: User metrics showed that one AI documentation site had a 70% drop-off rate on certain pages. After refactoring those pages to be more concise and task-oriented, the drop-off rate fell to 30%.
Managing the Maintenance Workload: Making It Sustainable
Documentation maintenance shouldnât be a heroic effort:
- Documentation sprints: Dedicate specific time periods just for documentation
- Rotation schedules: Share the maintenance responsibility across the team
- Priority framework: Focus on high-impact, high-visibility sections first
- Celebration and recognition: Reward good documentation maintenance
- Realistic planning: Budget time for documentation in each release cycle
The rotation win: One team implemented a weekly âdocumentation dutyâ rotation. Each person spent just one day focused on documentation improvements, but with 10 team members, their docs were continuously maintained with minimal individual burden.
Letâs Practice: Exercises to Build Your Version Control Muscles
Exercise 1: Design Your Version Control Strategy
The mission: Create a versioning plan for an evolving AI system.
Your adventure:
- Choose a real or hypothetical AI product (e.g., a sentiment analysis API thatâs rapidly evolving)
- Design a version scheme that aligns documentation with system versions
- Sketch a user interface for version selection
- Create a sample version compatibility matrix
- Write a template for version change notices
Reflection questions:
- How will users know which documentation version theyâre viewing?
- How many previous versions will you support, and why?
- Whatâs your strategy for handling deprecated features?
Exercise 2: Create a Documentation Maintenance Workflow
The mission: Develop a sustainable process for keeping docs current.
Your adventure:
- List the events that should trigger documentation updates
- Create a checklist for reviewing documentation accuracy
- Design a simple template for documentation change requests
- Outline roles and responsibilities for documentation maintenance
- Develop metrics for documentation freshness
Reflection questions:
- Who should be responsible for maintaining different parts of the documentation?
- How will you balance thoroughness with efficiency?
- What automation could make this process more reliable?
Exercise 3: Document a Model Update
The mission: Practice creating clear documentation for a model change.
Your adventure:
- Imagine a scenario where your image classification model has been updated
- Create a model changelog that explains:
- What specifically changed in the model
- Why these changes were implemented
- How performance metrics have changed
- What differences users might notice
- Include before/after examples showing the practical impact
- Add appropriate versioning information and compatibility notes
Reflection questions:
- How technical should this changelog be?
- What information would different user types (developers, business users, admins) need?
- How could you verify that the changelog is complete and accurate?
Your Documentation Time Machine Toolkit: Resources
Version Control and Docs-as-Code
- Docs as Code - The bible for treating documentation like source code
- DiĂĄtaxis Framework - A systematic approach to documentation organization
- Write the Docs Guide to Version Control - Practical version control approaches
Tools That Make Version Control Easier
- Docusaurus Versioning - Simple but powerful documentation versioning
- Sphinx Versioning Extension - Version control for Python documentation
- ReadTheDocs Versioning - Hosted documentation with excellent version support
- Semantic-Release - Automate version management
Guides and Best Practices
- Keeping Documentation Fresh - Practical strategies to fight documentation decay
- Machine Learning Governance - AWSâs guide to model versioning and governance
- Googleâs Developer Documentation Style Guide - The gold standard for technical writing consistency
Frequently Asked Questions About Version Control for AI Documentation
Get answers to common questions about managing documentation across multiple AI system versions, maintaining accuracy during rapid development, and implementing effective versioning strategies.
Documentation Versioning Essentials
AI documentation requires specialized versioning approaches because: 1) AI systems evolve through both deliberate updates and organic learning, meaning behavior can change without explicit code releases; 2) Model drift occurs as real-world data patterns shift over time, potentially making documentation inaccurate even when the model code hasnât changed; 3) Multiple versions of AI models often run simultaneously (for different user segments, A/B testing, or specialized domains); 4) Performance characteristics can vary significantly between versions, requiring detailed version-specific documentation; 5) Training data versions must be tracked alongside model versions to fully understand system behavior; 6) Backward compatibility is often limited, as new model versions may interpret inputs differently; 7) Algorithmic improvements can fundamentally change how the system processes information rather than just adding features; and 8) Regulatory requirements often mandate documentation of model lineage and version histories. Unlike traditional software where version changes typically mean added/modified features with predictable differences, AI system changes can fundamentally alter the systemâs behavior patterns, decision boundaries, and failure modesâall of which must be precisely documented for each version to prevent misuse and set accurate expectations.
The most effective alignment strategy combines several approaches: 1) Implement semantic versioning (Major.Minor.Patch) that reflects the scale of behavioral changes, not just technical updates; 2) Create documentation branches that perfectly mirror your model deployment branches; 3) Treat documentation as part of the release artifactâno model gets deployed without its corresponding documentation; 4) Implement automated âdoc-blockingââpreventing model releases if documentation hasnât been updated; 5) Use version-specific URLs (e.g., docs.ai-system.com/v2.1/) that remain stable even as newer versions are released; 6) Include version selectors in your documentation UI with clear indicators of which version the user is viewing; 7) Add âcompatibility matricesâ that show which features exist in which versions; 8) Document the expected behavioral differences between versions, not just feature changes; 9) Include timestamps for both model training and documentation updates to help users assess freshness; and 10) Implement automated tests that verify documentation accuracy against each specific model version. The most successful teams integrate documentation versioning directly into their MLOps pipeline, where documentation updates are triggered by model retraining events and verified before deployment. This prevents the common problem of models evolving more rapidly than their documentation can keep pace.
To efficiently manage documentation for multiple AI model versions: 1) Implement a single-source-of-truth content system where shared content is maintained once but published to multiple version documentation sets; 2) Create a modular documentation architecture with clear separation between version-specific content and evergreen content; 3) Use inheritance patterns where newer documentation versions automatically inherit from previous versions, only overriding what has changed; 4) Implement a content management system with robust version control features specifically designed for technical documentation; 5) Build automated difference detection that highlights what has changed between versions; 6) Create standardized templates for version-specific information like performance metrics, limitations, and use cases; 7) Establish a clear retirement policy for older version documentation with appropriate archival processes; 8) Implement automated testing that verifies examples work correctly in each supported version; 9) Use tagged metadata to clearly mark content with applicable version ranges; and 10) Deploy a continuous integration system that automatically builds and publishes documentation for each supported version. The most efficient approaches balance the trade-off between duplication (maintaining separate complete documentation sets for each version) and flexibility (being able to precisely document version-specific behavior). A hybrid approachâwhere structural elements and general concepts are shared while version-specific details are maintained separatelyâtypically provides the optimal balance.
Technical Approaches to Documentation Versioning
The most effective tools for versioned AI documentation include: 1) Docs-as-code systems like Sphinx or Docusaurus with built-in versioning support that integrate with Git workflows; 2) Documentation-specific version control systems like Paligo or Heretto (formerly easyDITA) that handle complex versioning requirements; 3) Component content management systems (CCMS) that support reusable content across versions; 4) Static site generators with versioning plugins such as VuePress or Jekyll with version support; 5) API documentation tools with versioning like Redocly or Stoplight; 6) Specialized AI documentation tools like Model Cards Toolkit that handle model-specific versioning needs; 7) Version-aware rendering systems that can dynamically display different content based on selected versions; 8) Automated screenshot tools that maintain version-specific UI images; 9) Documentation testing frameworks that verify accuracy across versions; and 10) CI/CD pipelines with documentation-specific stages that automate versioned documentation deployment. For AI documentation specifically, integration with MLOps tools is crucialâsolutions that can automatically extract model parameters, performance metrics, and version information directly from your ML pipeline. The ideal toolchain connects your model registry, experimentation platform, and documentation system, enabling automatic updates when models change. Organizations with mature practices typically implement a documentation toolchain that mirrors their development toolchain, with parallel version control, testing, and deployment processes.
To implement documentation-as-code for AI systems: 1) Store documentation in the same repositories as code, using formats like Markdown or reStructuredText that work well with version control; 2) Establish branch and merge strategies that keep documentation synchronized with corresponding code/model changes; 3) Implement automated documentation builds that are triggered by code changes and model retraining; 4) Create documentation linters that check for technical accuracy, terminology consistency, and version alignment; 5) Set up automated testing for documentation examples, ensuring they work with the specific AI model version; 6) Generate API documentation and parameter references directly from source code to ensure accuracy; 7) Extract model specifications, performance metrics, and benchmarks directly from model artifacts; 8) Implement documentation-specific continuous integration that verifies documentation quality before allowing merges; 9) Include documentation reviews as part of the regular code review process; and 10) Use feature flags for documentation that mirror those used in the code, allowing gradual rollout of documentation changes. The most effective implementations integrate with ML-specific workflowsâconnecting documentation generation to model training pipelines, experiment tracking systems, and model registries. This approach ensures documentation is always paired with the correct model version and reflects current model behavior. When documentation lives alongside code and follows the same processes, itâs more likely to stay current and accurate during rapid development cycles.
Key automation approaches for sustainable versioned AI documentation include: 1) Auto-generated model cards that extract specifications, parameters, and performance metrics directly from training runs; 2) Dynamic API reference documentation that uses code introspection to stay synchronized with implementation; 3) Automated version detection that warns users when theyâre viewing documentation for a different version than theyâre using; 4) Continuous integration for documentation that tests examples, verifies links, and validates technical accuracy; 5) Automated difference highlighting that shows what changed between versions, helping users transition; 6) Template-based content generation that creates consistent documentation structures across versions; 7) Scheduled documentation health checks that identify outdated content based on timestamp and access patterns; 8) Integration with experiment tracking systems that automatically document model iterations and performance changes; 9) Documentation analytics that identify which sections are most viewed and potentially need more frequent updates; and 10) Notification systems that alert documentation owners when underlying models or APIs change. The most advanced organizations implement âdocumentation observabilityââmonitoring documentation effectiveness and accuracy with the same rigor as application performance. This includes tracking documentation-related support tickets, monitoring failed documentation searches, and measuring documentation usage patterns to identify gaps. By treating documentation as a product with measurable quality metrics, teams can focus automation efforts on the highest-impact areas.
Managing Documentation Evolution
To effectively document model evolution and drift: 1) Create a formal model lineage system that tracks how each model version relates to previous versionsâincluding training data changes, architecture modifications, and hyperparameter adjustments; 2) Implement performance tracking over time with clear visualizations showing how metrics have changed across versions; 3) Document both intentional changes (retraining with new data) and organic drift (performance changes due to shifting real-world patterns); 4) Maintain a detailed changelog that explains not just what changed but why changes were made and their expected impact; 5) Establish a threshold system that triggers documentation updates when performance drifts beyond certain boundaries; 6) Create benchmark datasets that allow consistent comparison between versions and over time; 7) Include section-specific âlast verifiedâ dates showing when each documentation component was last confirmed accurate; 8) Document known triggers for performance changes, such as seasonal patterns or data distribution shifts; 9) Implement automated drift detection that generates documentation updates when significant changes are detected; and 10) Maintain a separate âmodel behavior changesâ section that highlights differences users might notice rather than just technical changes. The most comprehensive approach treats model documentation as a living historical recordânot just describing the current state but preserving information about how and why the model has evolved, which helps users understand behavioral patterns and predict future changes.
Best practices for documenting deprecated AI features include: 1) Implement a consistent visual system for deprecation noticesâsuch as warning banners or color-codingâthat immediately signals deprecation status; 2) Create a multi-stage deprecation lifecycle with clear documentation at each phase: âplanned for deprecation,â âdeprecated but supported,â and âremovedâ; 3) Provide explicit migration paths and alternatives for each deprecated feature, with step-by-step transition guides; 4) Include precise timelines for how long deprecated features will remain available before removal; 5) Document the rationale behind deprecations, helping users understand why changes are necessary; 6) Maintain archived documentation for removed features, clearly marked as historical; 7) Implement version-specific search that can find deprecated features but clearly marks them in results; 8) Provide code migration tools or scripts alongside documentation to assist with transitions; 9) Document potential risks or side effects of continuing to use deprecated features; and 10) Establish a notification system to proactively alert users of deprecations affecting features they frequently use. For AI systems specifically, deprecation documentation should address model behavior changes, not just API changesâexplaining how the removal or replacement of capabilities might affect overall system performance, bias characteristics, or decision boundaries. Unlike traditional software where deprecated features simply disappear, AI system changes may have subtle ripple effects on other functionalities that should be thoroughly documented.
To maintain documentation quality during rapid AI development: 1) Implement a âdocumentation definition of doneâ where no feature or model update is considered complete without updated documentation; 2) Create tight integration between documentation and development workflows, making documentation updates part of the same tickets/issues as code changes; 3) Establish a documentation triage system that prioritizes updates based on user impact, usage patterns, and criticality; 4) Adopt modular documentation architecture where components can be updated independently without requiring complete rewrites; 5) Implement documentation quality gates in your CI/CD pipeline that prevent deployment if documentation doesnât meet standards; 6) Distribute documentation responsibility across the team rather than relying on dedicated technical writers alone; 7) Create templates and checklists that standardize documentation updates, making them faster and more consistent; 8) Implement a regular documentation audit cycle that verifies accuracy independently from development cycles; 9) Use feature flags for documentation that mirror code feature flags, allowing documentation to be prepared before features are fully released; and 10) Adopt documentation monitoring that alerts teams when usage patterns suggest documentation-reality mismatches (such as high bounce rates or search failures on specific pages). The most successful organizations integrate documentation into their agile processes as a first-class deliverableâtracking documentation debt alongside technical debt and allocating specific capacity for documentation maintenance in each sprint.
Test Your Knowledge
Test your understanding of version control strategies for AI-ML documentation with this quiz!
Version Control for AI Documentation Quiz
According to the chapter, what makes maintaining AI documentation harder than traditional software documentation?
The Journey Continues: Whatâs Next?
In our next module, weâll explore documentation workflows and collaboration models for AI-ML teams. Youâll learn how to integrate documentation into your development process from the very beginningânot as an afterthought when everyoneâs already moved on to the next feature.
Remember: In the world of AI, your documentation is a living entity that should evolve alongside your systems. With the version control strategies weâve explored, youâre now equipped to ensure your users never again have to wonder, âWait, is this documentation even for the right version?â
Found value in the course?
Your support fuels my work!
Have an issue?
Help improve the course by reporting an issue
