Deliver Infrastructure and Software running on it Rapidly and Reliably at Scale

Table of contents

What is Infrastructure as Code
Key Principles
- Idempotency
- Immutability
Patterns and Practices
- Everything in Source Control
- Modularize and Version
- Documentation
- Testing
- Security and Compliance
- Automate Execution from a Shared Environment
Infrastructure as Code Pipeline
GitOps
Conclusion

Notes:
1. There is a certain level of Organizational maturity needed to use these Principles, Patterns, and Practices. This article is not focused on the cultural side of things but it is very important for the successful adoption of these.
2. Examples used in this article are using Terraform and AWS but these Principles, Patterns, and Practices are generic and can mostly be applied to other IaC tools like
Pulumi, CloudFormation, etc. …


Based on my experience leading Platform Engineering initiative at various organizations

I talked about what Platform Engineering(PE) is, when is it useful, and the challenges I have seen working with it in my previous article. In this article, we will go through solutions that have worked for my teams and me in resolving those challenges. I have led platform engineering teams at various companies of different sizes (from startup to enterprise), and these challenges and solutions are based on my experience.

Challenges

Let’s go through each of the challenges and what I have done to solve them.

#1: Avoid creating another silo

As mentioned in the previous article, one of the DevOps movement’s critical aspects is to reduce silos between various teams. Creating another team sounds like the opposite of that, and you should be careful about not creating another silo. …


What is it, When is it useful, and Challenges with it

Table of contents

What is Platform Engineering?
Platform Engineering Team
Who are the customers?
When is it useful?
Challenges with Platform Engineering
Conclusion

Caution: Platform is a widely used term to define various types of platforms, so be careful about using it. At one place, we chose not to use the term platform in the team name and just called it the Infrastructure team since there was another AI platform team. Another place we called it the Infrastructure Platform Team. Use the term that makes sense for your organization, but the concept remains the same. …


Deploying Machine Learning Systems to Production safely and quickly in a sustainable way

Table of contents

What is Continuous Delivery?
Continuous Integration
Continuous Delivery vs. Continuous Deployment
Machine Learning Workflow
How does Continuous Delivery help with ML challenges?
Data Management
- Automated Data Pipeline
Experimentation
- Training Code
- Training Process
Production Deployment
- Application Code
Bringing it all together
People
Conclusion

Image for post

Most of the principles and practices of traditional software development can be applied to Machine Learning(ML), but certain unique ML specific challenges need to be handled differently. We discussed those unique “Challenges Deploying Machine Learning Models to Production” in the previous article. …

About

Adarsh Shah

Engineering Leader, Independent Consultant, Coach, Public Speaker, Hands-on Architect | Website: shahadarsh.com

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store