If your company's development lifecycle is struggling to run efficiently, it might be time for you to consider platform engineering and an in-house platform for developers.
Modern software continues to become increasingly complex. Unlike the monolithic software of yesterday, today's software is more flexible, agile, high-performance, and capable of meeting ever-increasing demands without requiring significant hardware scaling by the enterprise.
These modern application stacks are made possible by virtual machines, containers, and the cloud. But along with these technologies come significant complexities. Given the significant challenges that modern technology is capable of overcoming, these complexities are often considered an acceptable addition to the modern workflow. After all, when your company needs to automatically scale deployments based on usage, you certainly won't be able to do so with a traditional monolithic application.
This is especially true for companies looking to undergo a major digital transformation, where there may be a lot of work to be done.
For all these reasons, your company may need a more modern approach to software development. Such an approach is platform engineering.
What is platform engineering?
Platform engineering is an approach to designing and developing workflows that enable self-service capabilities for programmers and engineers. With platform engineering in place, your teams will have access to an internal developer platform (IDP) that includes everything needed to manage the software development lifecycle.
The most important thing about platform engineering is that not all teams or companies have the same development/deployment process. Because of this, each tool in the toolkit can vary depending on your needs. With this in mind, your teams and team leaders will have to work together to build the exact toolset and services needed to make your workflow run smoothly, effectively, and efficiently.
One of the problems with modern development is that software engineers are now tasked with handling multiple tasks using different tools. There's Docker, Kubernetes, Helm, Terraform, Ansible, Puppet, container manifests, CI/CD tools, and a host of frameworks and libraries. Imagine having to keep all these moving parts together so that every developer/designer has access to what they need, when they need it.
With developers having to take on more tasks, they must be able to claim ownership of each step of the process. They need access to cluster environments, development tools, and automation services without having to go through different team leads for approval. Unlike the way things were handled previously, where you might have teams dedicated to specific tasks in the lifecycle, developers and engineers are now tasked with taking care of everything.
If these developers and engineers don't have access to everything they need, the work won't be completed in a timely manner. These developers must be able to build and deploy their software end-to-end. However, when you start to consider everything, you realize that such an approach can be quite problematic. Unless your company has a considerable budget to spend, reaching a state of true DevOps can be an impossible task.
To avoid these challenges and greatly improve the process, self-service IDPs are one of the best ways to achieve true DevOps, and for this to happen, you need to employ platform engineering to create the ideal IDP for your company/project. This IDP will include everything your development teams need and will be organized in a way that best matches your team's prescribed workflow.
By employing platform engineering, you greatly improve developer productivity (while improving your overall experience) by working with reusable tools, automated operations, and self-service capabilities. Each component of the IDP must be highly configurable and flexible. With a solid IDP in place, your developers will have access to standardized, reliable tools that will not only simplify the process but also make it easier to repeat. The ultimate goal is a frictionless environment that enables your teams to have everything they need, when they need it.
Typical tools used in platform engineering
Each IDP will be different depending on the needs of your development teams, business, and workflow. Here is a short list of tools you should consider adding to your PDI:
GitHub
GitHub is a web-based host for software development and version control. No development team is complete without this tool. And with GitHub, your developers can work from the command line or a supported GUI. Either way, all your teams will work from a single source of truth.
To enjoy
If you want to add AI/ML to the mix, Leverage is the tool you need. This piece of the IDP puzzle should easily integrate and manage your technology stack with powerful orchestration capabilities.
Kubernetes
If you want to containerize your deployments, you will need an orchestration tool. With Kubernetes you get automation of things like container configurations and scaling, container isolation for greater security, and automatic deployment across multiple nodes.
Terraforma
Terraforma is a crucial piece of the PDI stack, as infrastructure is often thought of as de facto code that facilitates the provisioning and management of infrastructure. With Terraform, your teams will be able to manage your infrastructure across multiple platforms and even automate security based on centralized policies.
Ansible
Ansible is one of the most used configuration management platforms on the market. With Ansible, you create playbooks for states that are distributed to machines to ensure everything is always in a predictable state. You can install software on an entire network of remote machines from a single system. Ansible can be used to manage bare metal, cloud-hosted instances, and virtual machines.
Gate
If your teams are tasked with managing multiple Kubernetes or Docker clusters on different nodes, you'll want the power of a user-friendly, web-based GUI in your IDP. The best tool for this job is Gate which makes it possible to manage all aspects of deployment and management in containers.
Docker/Docker Swarm
One thing you'll learn very quickly is that Docker and Docker Swarm make working with containers considerably easier than working with Kubernetes. While you may not get the advanced management tools, with the addition of Portainer, much of this problem is alleviated.
drone
Drone is a tool for creating and testing the automated side of your workflow. Drone integrates with GitHub, makes it easy to create custom plugins, and can work with virtually anything running inside a Docker container. If you need automation, you will need the Drone to test it.
It will take time
One of the first realities you must accept is that building your IDP will take time. Right off the bat, you should figure out the pieces that can be assembled to best meet your workflow needs. If you're not sure where to start, I would start with the deployment tool. Decide whether you want to work with Kubernetes or Docker (or will you need both). Once you decide this, you can start working through the process and decide which software you will need for each step of the process. While some of these tools can be obtained for free, you will also need to consider what kind of budget you have for your PDI and work from there. If you cannot purchase all the necessary tools, are there open source equivalents that can be substituted to reduce costs?
Additionally, you should consider your current workflow and which steps/phases tend to bottleneck the process. If you can identify the problems, you can resolve them when setting up your PDI.
With the right amount of upfront work, platform engineering can help your company create a PDI that will help make the development/deployment process more efficient, cost-effective, and reliable.
