12 Factor Apps
Overview
A group of people within Heroku developed the 12 Factors in 2012. This is essentially a manifesto describing the rules and guidelines that needed to be followed to build a microservices based cloud-native application. The manifesto is currently hosted here. This methodology presents a thought process on how to develop applications and is complimentary to other methodologies and thought processes such as the reactive manifesto.
The 12 factors are
- Codebase - One codebase tracked in revision control, many deploys.
- Dependencies - Explicitly declare and isolate dependencies.
- Configuration - Store configuration in the environment.
- Backing Services - Treat backing services as attached resources.
- Build, release, run - Strictly separate build and run stages.
- Processes - Execute the app as one or more stateless processes.
- Port binding - Export services via port binding.
- Concurrency - Scale out via the process model.
- Disposability - Maximize robustness with fast startup and graceful shutdown.
- Dev/prod parity - Keep development, staging, and production as similar as possible.
- Logs - Treat logs as event streams
- Admin processes - Run admin/management tasks as one-off processes
Codebase
“A twelve-factor app is always tracked in a version control system… A codebase is any single repo (in a centralized revision control system like Subversion), or any set of repos who share a root commit (in a decentralized revision control system like Git).”
One codebase tracked in revision control, many deploys. There can only be one codebase per application. The codebase must be managed by a version control system. Various deploys are generated from this codebase, each one for a different environment—development, staging, and production. This looks too simplistic and non nonsensical but if you think about it in the world of SOA and microservices we have to think through our version control strategies. Each service should be maintained as its own codebase and should be version controlled independently.
However there is no one size fits all with codebase strategies and teams move from Mono repo to Multi repos based on various factors. Mono Repo is where all Microservices are housed within a single repository. In multi repo each microservice codebase is tracked in its own independent repo.
Dependencies
“A twelve-factor app never relies on implicit existence of system-wide packages.”
Dependencies used by a project and their versions must be explicitly declared and isolated from code. Explicitly stating versions results in lower compatibility issues across environments. This also results in better reproducibility of issues occurring in specific version combinations. Dependencies not only include packages but also platforms, SDK’s etc. Package dependencies can be manged using package management tools like Nuget, NPM etc. Container technology simplifies this further by explicitly specifying all installation steps. It also specifies versions of base images , so that image builds are idempotent.
Application configuration
“Apps sometimes store config as constants in the code. This is a violation of twelve-factor, which requires strict separation of config from code. Config varies substantially across deploys, code does not.”
Application configuration, that differ across environments such as external dependencies, databases, credentials, ports etc are manifested at runtime. This configuration should not be hard coded in the source code but dynamically modifiable from outside of the application. The use of environment variables that can be injected when deploying an application in a specific environment is recommended.
Backing Services
“The code for a twelve-factor app makes no distinction between local and third-party services…Each distinct backing service is a resource.”
Databases, API’s and other external systems that are accessed from the application are called resources. The application should be abstracted away from its external resources. These resources should be attached to the application in a loosely coupled manner. They should be replaceable by different instances without any impact to the application. Backing services should be abstracted into individual components with clean interfaces.
Build, Release, Run
“The twelve-factor app uses strict separation between the build, release, and run stages.”
A single codebase is taken through the build process to produce a compiled artifact. The output of the build stage is combined with environment specific configuration information to produce another immutable artifact, a release. Each release is labelled uniquely. This immutable release is then delivered to an environment (development, staging , production, etc.) and run. If there are issues this gives us the ability to audit a specific release and roll back to a previously working release.
Processes
“Twelve-factor processes are stateless and share-nothing.”
All processes and components of the application must be stateless and share-nothing. An application can create and consume transient state while handling a request or processing a transaction, but that state should all be gone once the client has been given a response. All long-lasting state must be external to the application and provided by backing services. Processes come and go, scale horizontally and vertically, and are highly disposable. This means that anything shared among processes could also vanish, potentially causing a cascading failure.
Port Binding
“The twelve-factor app is completely self-contained and does not rely on runtime injection of a webserver into the execution environment to create a web-facing service.”
A twelve-factor app is fully self-contained and does not depend on the use of an external server, such as IIS or NGNIX, to be exported as a service.It is self-contained and is never injected into any kind of external application server or container. A twelve-factor app must export the HTTP service by port-binding, meaning that the application also interfaces with the world via a URL. The application might run as http://localhost:5001 on a developer’s workstation, and in QA it might run as http://164.132.1.10:5000, and in production as http://service.company.com. An application developed with exported port binding in mind supports this environment-specific port binding without having to change any code.
Concurrency
“In the twelve-factor app, processes are a first class citizen…The process model truly shines when it comes time to scale out.”
Applications should scale out using the process model. Elastic scalability can be achieved by scaling out horizontally. Rules can be setup to dynamically scale the number of instances of the application/service based on load or other runtime telemetry. Stateless, share-nothing processes are well positioned to take full advantage of horizontal scaling and running multiple, concurrent instances.
Disposability
“The twelve-factor app’s processes are disposable, meaning they can be started or stopped at a moment’s notice.”
Processes are constantly created and killed on demand. An application’s processes are disposable, and can be started or stopped rapidly. An application cannot scale, deploy, release, or recover rapidly if it cannot start rapidly and shut down gracefully. Shutting down gracefully implies saving the state if necessary, and releasing the allocated computing resources.
Dev/Prod Parity
“The twelve-factor app is designed for continuous deployment by keeping the gap between development and production small.”
All environments should be maintained to be as similar as possible.
Logs
” A twelve-factor app never concerns itself with routing or storage of its output stream.”
Logs should be treated as event streams.Logs are a sequence of events emitted from an application in time-ordered sequence. A cloud-native application writes all of its log entries to stdout and stderr.You can use tools like the ELK stack (ElasticSearch, Logstash, and Kibana), Splunk etc to capture and analyze your log emissions. Applications should be decoupled from the knowledge of log storage, processing, and analysis. Logs can be directed anywhere. For example, they could be directed to a database in NoSQL, to another service, to a file in a repository, to a log-indexing-and-analysis system, or to a data-warehousing system.
Admin Processes
“Run admin/management tasks as one-off processes.”
Maintenance tasks, such as script execution for data migration, initial data seeding and cache warming should be automated and performed on time.