Peanut Butter and Chocolate: Azure Functions CI/CD Pipeline with AWS CodeCommit (Part 5 of 6)

Part 5

Sorry for the delay between part 4 and 5! I was at the Microsoft MVP Summit this past week and didn’t have time to devote towards updating. This series is nearing completion with just a few more parts to go.

In Part 4 we published the AWS Lambda and created the AWS KMS Key that will be used for encrypting and decrypting secrets. In Part 5 we will configured the AWS CodeCommit repository trigger to invoke the AWS Lambda and encrypt our secrets to store in in the cc2af.yml file.

Series Table of Contents

• Part 1 of 6: Project Introduction and Overview
  
• Part 2 of 6: Project Settings, Authentication, Azure Function App Deployment, and AWS Code Commit Deployment
  
• Part 3 of 6: Creating AWS IAM User for Git, Generating HTTPS Git Credentials, Configuring Azure Web App External Git Deployment, and Initial Manual Deployment
• Part 4 of 6: Create AWS IAM Role, Deploy the C# .NET Core 2.0 AWS Lambda, and Create an AWS KMS Key
• Part 5 of 6 (This Post): Add AWS CodeCommit Trigger and Encrypt Secrets with AWS KMS
• Part 6 of 6: Generate Configuration File, Automatic deployment from AWS CodeCommit to Azure Functions, Trigger the Azure Function from PowerShell, and Series Conclusion

Peanut Butter and Chocolate: Azure Functions CI/CD Pipeline with AWS CodeCommit (Part 4 of 6)

Part 4

In Part 3 we successfully made the first glue between Azure Functions and AWS CodeCommit by making it possible to manually trigger the Azure Functions Web App to pull from the AWS CodeCommit repository. Obviously, a manual pull is not ideal. It is certainly not a Continuous Delivery.

In Part 4 we lay the groundwork for the 2nd piece of glue between Azure Functions and AWS CodeCommit. In order to automatically trigger a pull AWS CodeCommit from Azure Functions, we need an AWS Lambda. AWS Lambda and Azure Functions are somewhat analogous. They serve almost identical purposes in their respective clouds. We also need to create a KMS key that will be used for encrypting and decrypting secrets.

Series Table of Contents

• Part 1 of 6: Project Introduction and Overview
  
• Part 2 of 6: Project Settings, Authentication, Azure Function App Deployment, and AWS Code Commit Deployment
  
• Part 3 of 6: Creating AWS IAM User for Git, Generating HTTPS Git Credentials, Configuring Azure Web App External Git Deployment, and Initial Manual Deployment
• Part 4 of 6 (This Post): Create AWS IAM Role, Deploy the C# .NET Core 2.0 AWS Lambda, and Create an AWS KMS Key
• Part 5 of 6: Add AWS CodeCommit Trigger and Encrypt Secrets with AWS KMS
• Part 6 of 6: Generate Configuration File, Automatic deployment from AWS CodeCommit to Azure Functions, Trigger the Azure Function from PowerShell, and Series Conclusion

Peanut Butter and Chocolate: Azure Functions CI/CD Pipeline with AWS CodeCommit (Part 3 of 6)

Part 3

In Part 2 we created the Azure Functions Web App and the AWS CodeCommit repository. In Part 3 we will make the initial deployment from AWS CodeCommit to Azure Functions. To do that we need to create an AWS IAM User Account, grant it access to the CodeCommit repository, generate HTTPS Git Credentials for the user, and configure the Azure Functions Web App external git deployment. By the end of this post, we will be able to manually deploy from AWS CodeCommit to Azure Functions on demand. This is a critical step to make automating the process possible.

This part will be short and sweet. I want to keep the relevant pieces together regardless of their length.

Series Table of Contents

• Part 1 of 6: Project Introduction and Overview
  
• Part 2 of 6: Project Settings, Authentication, Azure Function App Deployment, and AWS Code Commit Deployment
  
• Part 3 of 6 (This Post): Creating AWS IAM User for Git, Generating HTTPS Git Credentials, Configuring Azure Web App External Git Deployment, and Initial Manual Deployment
• Part 4 of 6: Create AWS IAM Role, Deploy the C# .NET Core 2.0 AWS Lambda, and Create an AWS KMS Key
• Part 5 of 6: Add AWS CodeCommit Trigger and Encrypt Secrets with AWS KMS
• Part 6 of 6: Generate Configuration File, Automatic deployment from AWS CodeCommit to Azure Functions, Trigger the Azure Function from PowerShell, and Series Conclusion

Peanut Butter and Chocolate: Azure Functions CI/CD Pipeline with AWS CodeCommit (Part 2 of 6)

Part 2

In Part 1 I gave an overview of the Peanut Butter and Chocolate Project. In Part 2 I will cover the required PowerShell modules for the deployment, the settings used in the project, authenticating tp Azure and AWS, deploying the Azure Function App, and Deploying the AWS CodeCommit repository. I also demonstrate how to verify the resource deployments with Pester.

As a reminder, you can obtain the project code from https://github.com/markekraus/PeanutButterChocolate

Series Table of Contents

• Part 1 of 6: Project Introduction and Overview
  
• Part 2 of 6 (This Post): Project Settings, Authentication, Azure Function App Deployment, and AWS Code Commit Deployment
  
• Part 3 of 6: Creating AWS IAM User for Git, Generating HTTPS Git Credentials, Configuring Azure Web App External Git Deployment, and Initial Manual Deployment
• Part 4 of 6: Create AWS IAM Role, Deploy the C# .NET Core 2.0 AWS Lambda, and Create an AWS KMS Key
• Part 5 of 6: Add AWS CodeCommit Trigger and Encrypt Secrets with AWS KMS
• Part 6 of 6: Generate Configuration File, Automatic deployment from AWS CodeCommit to Azure Functions, Trigger the Azure Function from PowerShell, and Series Conclusion

Peanut Butter and Chocolate: Azure Functions CI/CD Pipeline with AWS CodeCommit (Part 1 of 6)

Source: 1981 Reese's Peanut Butter Cup Advertisement

Intro

This blog series will cover a Proof of Concept (POC) Project for creating a PowerShell-based Azure Functions CI/CD pipeline where the code is stored in AWS CodeCommit git-based version control system. The pipeline will be created and deployed using Windows PowerShell 5.1. Every step of the pipeline deployment process will be verified with Pester tests. The result of the project will be the ability to push changes to an AWS CodeCommit repository and those changes will be automatically deployed to Azure Functions.

This blog series is targeted at intermediate level PowerShell users and basic PowerShell concepts will not be described in detail. Also, this series will require some basic understanding of both Azure and AWS clouds and their PowerShell based management. Git and C# .NET Core are also leveraged in this project but they will not be covered in depth as this is a PowerShell-centric blog. Readers need only be familiar with basic concepts of git and C#.

Series Table of Contents

• Part 1 of 6 (This Post): Project Introduction and Overview
  
• Part 2 of 6: Project Settings, Authentication, Azure Function App Deployment, and AWS Code Commit Deployment
  
• Part 3 of 6: Creating AWS IAM User for Git, Generating HTTPS Git Credentials, Configuring Azure Web App External Git Deployment, and Initial Manual Deployment
• Part 4 of 6: Create AWS IAM Role, Deploy the C# .NET Core 2.0 AWS Lambda, and Create an AWS KMS Key
• Part 5 of 6: Add AWS CodeCommit Trigger and Encrypt Secrets with AWS KMS
• Part 6 of 6: Generate Configuration File, Automatic deployment from AWS CodeCommit to Azure Functions, Trigger the Azure Function from PowerShell, and Series Conclusion

Multipart/form-data Support for Invoke-WebRequest and Invoke-RestMethod in PowerShell Core

Pictured: A packet capture of a current build of PowerShell Core submitting a multipart/form-data POST request from Invoke-WebRequest.

Intro

Over the past few months I have been donating a generous portion of my spare time to help improve the Web Cmdlets (Invoke-WebRequest and Invoke-RestMethod) in PowerShell Core. This is partly because I want and need certain functionality for both personal and work related projects. It is also because I have had some minor gripes about these Cmdlets for some time.

One common ask I have seen repeated in just about every PowerShell forum is multipart/form-data support. It seems like a reasonable thing to ask when there are many endpoints that will only work with a multipart/form-data submission. There is an open issue (#2112) on the PowerShell GitHub echoing the same request. It was brought to my attention and I decided to give it a serious look.

The result is that PowerShell Core now has partial multipart/form-data support in both Web Cmdlets. This change didn't make the cut for 6.0.0-beta.7 but it will be available starting in 6.0.0-beta.8 and is available now if you build it manually or grab the latest nightly build.

This blog will cover some of the challenges involved in supporting multipart/form-data, how to make use of this new feature, and about future plans for additional support.

Because typing multipart/form-data is annoying, I will be shortening it to just multipart. Please don't let this be mistaken for other multipart submission methods.

Also, I will be referring collectively to Invoke-WebRequest and Invoke-RestMethod as Web Cmdlets. In this case, there is no need to call out each command as they offer the same base functionality for multipart support.

Porting PSRAW to PowerShell Core: Lessons Learned

Intro

I took a significant break from commits to my PSRAW project. I have spent that time learning more about Open Source projects and Object Oriented Programming. Before I could move the project forward I had some architectural decisions to make and I don't have quite enough knowledge to make those decisions yet, but I'm getting there.

On July 14th, this blog from Microsoft dropped a bit of a bombshell on the PowerShell community by making it clear the path forward is in PowerShell Core.Windows PowerShell will still be developed/maintained, but the primary focus will be PowerShell Core. There was also a call to test out PowerShell Gallery modules. I put off the leap to Core for awhile, but it seemed that now was the time. My module is still young and flexible and I suspected that most of it would work on Core.

Shortly after that blog post, I  unleashed the Kraken and created a new branch in my local PSRAW repo named CoreRefactor, installed PowerShellCore 6.0.0-beta.4, switched VS Code to use PowerShell Core for the integrated terminal, and fired up my pester tests. Thus flowed a glorious sea of red failures and errors. This kicked off 2 weeks of refactoring. I wanted to share what I have learned from the experience.

At the time of this post PowerShellCore 6.0.0-beta.5 has just been released. I just completed my test against it. So everything in this post is at least relevant to that release.