Write The FAQ ‘n Manual (Part 1)

Automated Documentation in a CI/CD Pipeline for PowerShell Modules with PlatyPS, psake, AppVeyor, GitHub and ReadTheDocs

Part1: Intro, Pre-reqs, Components and Build Workflow 


One of my least favorite parts of IT is documentation. It takes a significant amount of time to document things, sometimes longer than the actual work you are documenting. I’m not alone. I hear complaints from all levels about how much of a pain and bore documentation is. However, we all depend on it. The moment things don’t go as planned or expected we start googling for solution and digging through documentation hoping to find the answer. So, while documentation is absolutely evil, it is a necessary evil.

With PowerShell, and other programming languages, there is an added burden of needing to document in more than one place. For example, you may need in-code documentation for what you are doing and why, programmatic help systems (like PowerShell’s comment based help), external help, internal help, and online documentation. It can get overwhelming and will often dissuade people from documenting their code at all or at the very least result in their only documenting in one place.
I recently published my PSMSGraph module to the PowerShell gallery. One of my pet peeves about many gallery modules is that there is just no documentation. Some don’t even include comment based help or any useful comment based help. You are left with a trial-and-error based approach to using them. I wanted to avoid that for anything I publish to the gallery. I’m not trying to put anyone down, but I just wanted to make my own module easy, more accessible, and at least somewhat documented.

Personally, a year ago, I promised myself to commit to comment based help on everything. Every script and function I write now include comment based help. I really didn’t want to duplicate my efforts in documentation. Since I always have comment based help I began searching for a method of converting that to markdown and I stumbled across a few scripts until I found PlatyPS. For my ConnectReddit module I had very manual process where I ran a script to generate markdown documentation and then it had to be pushed to the separate wiki repo. It was painful and I hated it. Since PSMSGraph is now in a CI/CD pipeline, I spent some time integrating PlatyPS into my pipeline. Now I don’t even need to think about the documentation part outside of keeping my comment based help updated.

This 4 part blog series will cover how to implement this in your pipeline. Once done you will have automated function online documentation as part of your pipeline as well as an automatically managed Change Log based on your Release notes.



Before you Begin

Please note that this blog series is not intended as an introduction to PowerShell. This blog is intended for those already familiar with PowerShell. 

This blog series builds on the work of Kevin Marquette and Warren Frame. They both have excellent blogs on how to implement a CI/CD pipeline for module development. This blog is written with the assumption that you are familiar with GitHub, AppVeyor, and psake. If you want to learn those things, please stop here and go read their blogs. 

Required reading:
Comment based help is essential to this process. For a refresher or a primer, you can run 
Get-Help about_comment_based_help
or visit the following URL
Make sure you have a decent understanding of those concepts and technologies before moving on with this series or you are sure to be lost. I will try my best not to rehash anything covered in their post except where my process or architecture differs.

Also, I recommend taking a primer on both markdown and YAML. I don’t claim to be an expert on either, but some basic familiarity with them will be needed for this.
Finally, you might want to look at the PSMSGraph project and you will definitely want to look at the AutoDocumentsExample project that I will be using for this series. With the PSMSGraph project you can see the entire build process from start to finish and how it ends up looking in the PowerShell Gallery.




Before I get into the nitty-gritty I figured it was best to name and describe the basic components that will be used to accomplish the goal of a CI/CD pipeline with automated documentation building. If you have done the “required reading”, some of this will be a review for you.



PlatyPS is a PowerShell module for generating documentation. For our purposes, it has a very convenient capability of creating markdown help documents for an entire module. It has a lot of other features for external documentation that I will want to look at in the future.



AppVeyor is a CI/CD build service. I like to think of it as a cloud Jenkins. It builds, tests, and deploys code. It also happens to be free for Open Source projects. Best of all, it supports PowerShell 5 which will be used to run the psake build.



psake is a DSL build automation module for PowerShell. It makes it easy to create build automation processes. It makes it easier to code your process as a set of tasks and dependencies. The /psake.ps1 in the project is where most of the magic happens. It is run in the AppVeyor environment and makes all our calls to the PlatyPS functions as well as recommitting our changes



ReadTheDocs is a community supported documentation build and host service. It supports mkdocs which turns markdown documents into HTML web pages. It supports a range of other documentation engines, but since we are using PlatyPS to generation in markdown, we will make use of the mkdocs engine. If you use their service, please consider contributing time or money. You can find more details about ways to contribute on their website.



If this is the first you have ever heard of GitHub, this blog post is not for you. The role GitHub obviously serves here is the public version control repository for our code. When we commit to the repo it will fire off the webhooks for both AppVeyor and ReadTheDocs which will automagically fire up the build pipeline.



Overview of the AutoDocumentationExample Repository

If you read the “required reading” this structure should look mostly familiar to you. I will go over the pieces that are different or standout in this configuration. For convenience, I will used / to refer to the top level of the project folder and refer to all files as their path name from /.



Our appveyor.yml file differs slightly in that we are also including an access_token secure environment variable. This is a GitHub access token you will need to generate and then convert to a secure string in your AppVeyor account. This is used to write our changes from the build in AppVeyor back to the repo. This can be named anything but  /psake.ps1 will need to be updated to reflect whatever name is given.



RELEASE.md is where we will put our Release Notes for the current release. As the .md file extension indicates, the file will be in markdown. There will be more details on what to put here later in the Release Preparation section. This file will be copied to the /docs/ folder and added as the ReleaseNotes field on the module manifest. This means that the text here will show up on the ReadTheDocs site as well as the PowerShell Gallery. psake also prepends this to the /docs/ChangeLog.md file



In Kevin’s blog, he brought this up but said he would go into detail later. Basically, this is used by ReadTheDocs’s mkdocs engine to configure and execute the documentation build process. For our project, we do not write to this directly. In fact, when starting your project, you do not need to include this. psake will overwrite or create this file based on what is in /header-mkdocs.yml , the release notes, the change log, and the function documentation generated by PlatyPS.



This is another new file. As stated above this is what will be used to generate /mkdocs.yml



This file is not new, but, it is different from other projects. There are 3 tasks that are of importance to this blog: Build, BuildDocs, and PostDeploy. These are where the magic happens for managing release notes, change log, documentation building, and committing build changes back to the repo.



This is the folder where the markdown files used by ReadTheDocs will live.



This serves as the landing page and “Home” page for the ReadTheDocs site



This copy of the /RELEASE.md is generated by psake.



This is the automatically maintained change log. Basically, this has /RELEASE.md prepended to it every build.


/functions/ and /functions/*.md

The function documentation generated by PlatyPS is stored in this folder, one per function in the module.



The Help.Tests.ps1 for this project includes verifying the function has a HelpUri defined and at least one related link.



Build and Documentation Build Flow

Now that the component and project architecture is out of the way, let’s talk about the project flow that is used here. My flow is probably not ideal and will probably be reworked over time. But, it’s functional enough for now that those looking to add automated documentation to their pipeline could likely drag and drop this into their project.  The basic flow is this:
  1. Modify code in project 
  2. Update comment based help for all public functions
  3. Erase contents of /RELEASE.md (left from the last release) add the changes made in this release
  4. Git add 
  5. Git commit
  6. Git push
  7. AppVeyor webhook triggered (ReadTheDocs is also triggered but this document build will be overwritten once AppVeyor finishes)
  8. /Build.ps1 sets environment and runs /psake.ps1
  9. /psake.ps1:
    1. Initializes
    2. Runs quick unit tests
    3. Build:
      1.  Populates functions in module manifest
      2.  Populates aliases in module manifest
      3.  Version bumps in module manifest
      4.  Prepend version and date to /RELEASE.md
      5.  Prepends /RELEASE.md to /doc/ChangeLog.md
    4. Runs all tests (unit tests are re-run to ensure build changes have not affected them)
    5. BuildDocs:
      1. Imports the freshly built module
      2. Grabs content of /header-mkdocs.yml
      3. Copies the /RELEASE.md to /docs/RELEASE.md
      4. Runs New-MarkdownHelp from PlatyPS to build the function documentation in /docs/functions/
      5. Generates the /mkdocs.yml based on the results of the above steps.
    6. Deploy (if triggered by !deploy in the commit)
    7. PostDeploy:
      1. Commits all the changes made by the build back to the repo with [ci skip]
      2. Creates a GitHub Release if !deploy is in commit.
  10. GitHub repo is updated with build changes
  11.  ReadTheDocs webhook is triggered (AppVeyor ignores because of the [ci skip] in the commit message)
  12. ReadTheDocs builds the documentation website
  13. Git Pull (You need to pull the changes made from the build process back into your local Git repo).
Continue to Part 2