Ephemeral environments offer developers a tool to solicit feedback from stakeholders, verify features and bug fixes before releasing to production, and perform automated end-to-end or performance testing. Equipped with Infrastructure-as-code (IaC), it’s quite easy to launch a new environment. However, teams face many challenges when setting up and managing ephemeral environments. This article goes deep into various use cases, benefits, challenges, and pitfalls of using ephemeral environments.
## What Are Ephemeral Environments?
Ephemeral environments are environments that mimic traditional environments—such as development, staging, and production—yet exist only temporarily. Also known as Preview Environments, Review Apps, or On-demand Environments, they offer a sandbox for testing, demos, and reviews, without the permanence or resource commitment of their traditional environments. These environments provide a dynamic, flexible approach to application development and deployment.
## Use Cases
Ephemeral environments have a wide array of applications:
- Testing: From end-to-end, performance, and security, to infrastructure testing, ephemeral environments offer an isolated, real-world scenario for rigorous validation of applications.
- Demos: They are ideal for presenting new features to stakeholders or partners, providing a comprehensive, hands-on experience without affecting production systems.
- Sales: Often, customers want to see how a product will work with their data before signing a contract (especially an annual contract). When automated properly, ephemeral environments provide a cheap, yet immersive buying experience for sales prospects.
## Benefits
The adoption of ephemeral environments brings numerous benefits to the delivery process, a critical driver for improving developer productivity and software quality.
### Improve developer productivity
Ephemeral environments significantly enhance developer productivity. They offer a secure and isolated area for testing new features or rectifying bugs, without the risk of destabilizing the existing system. This encourages more experimental and rapid innovation. As these environments are temporary and can be discarded post-use, developers are freed from the tasks of maintenance or cleanup, permitting them to concentrate primarily on development. Ephemeral environments can be launched quickly, offering developers instant feedback and accelerating the development process. Importantly, for teams unable to run the application on their local machines, ephemeral environments provide a valuable means for testing modifications prior to merging.
### Facilitate Isolated Experimentation
Ephemeral environments provide an excellent platform for isolated experimentation. They give developers a separate space where they can test new features or changes without risking the stability of the main codebase. This is particularly useful when working with cloud services, as it allows developers to quickly stand up services, ensure proper integration, and test the functionality without impacting the production environment. As such, ephemeral environments can be a critical tool for experimentation and innovation, facilitating a more iterative and agile development process.
### Enhance Testing
On-demand creation of environments enables testing on dedicated resources instead of shared infrastructure, leading to more stable testing conditions. This reduces distractions for the development team, who might otherwise spend time resolving issues caused by misconfiguration or data synchronization problems. Some teams impose code freezes on the staging environment to improve their testing quality, but this can disrupt productivity. Ephemeral environments eliminate the need for such code freezes.
### Mitigate configuration drift
Ephemeral environments allow software teams to embrace the concept of "Immutable Infrastructure". This implies that once an environment is set up, its state or configuration doesn't change over time. Instead of making modifications to an existing environment, a new one is created to meet the new requirements. This eliminates "configuration drift", which refers to the deviation of an environment's configuration over time.
By using ephemeral environments, every deployment happens in a fresh, identical environment. They mirror the exact configuration of the production environment, and any changes are discarded after use. This ensures consistency across all deployments, making the software more reliable and predictable.
Furthermore, it simplifies troubleshooting. Since there's no drift, no time is wasted identifying whether a problem is due to a configuration change. Errors can be traced back to code changes, making it easier to identify the root cause and resolve the issue. Additionally, Immutable Infrastructure reduces the risk of unauthorized access or changes, enhancing security.
## Challenges
If ephemeral environments provide massive advantages, why aren’t software teams adopting ephemeral environments into their workflows? In reality, ephemeral environments bring a set of hurdles that can be especially distracting to solve for smaller software teams or teams without strong infrastructure experience.
### Provisioning Time
Every environment is composed of many interconnected infrastructure components. Notably, launching a new virtual network, container clusters, and managed databases takes a significant amount of time. Even when fully automated, this process could take up to 1h. Without addressing this situation, developers will not adopt it as it is too cumbersome.
### Data Seeding
Every app needs data to be valuable. Some teams have a set of mock or seed data that is populated during environment creation. Others rely on a dump of production data into other environments. Each comes with its own set of challenges. If using mock or seed data, it requires discipline and maintenance to ensure the data is representative. If pulling from production, you’ll need to fuzz data — replace pieces of data to prevent exposure of sensitive consumer information, credentials, and customer information. This requires proper network access and automation to clone a production database into a non-production network.
### Subdomain Management
Most teams configure the subdomains for dev, staging, and prod manually. However, ephemeral environments require that your orchestration dynamically provision subdomains for each preview environment. To do this, you need to dynamically create DNS records using a naming scheme that is unique but must also be valid (i.e. avoid bad characters in a subdomain). Additionally, we need to configure TLS/SSL certificates for encrypted communication between the app and the user. (Many browsers require HTTPS)
### Configuration Management
Managing configuration across dev, staging, and prod environments is difficult enough. Ephemeral environments require extra attention because most apps rely on a set of credentials, API keys, configuration flags, and data sources that don’t act the same between environments. For example, one database may require dynamic access controls while another configuration value is configured manually to a static source. As well, you may use the same API key across all ephemeral environments for a third party. Even more, what if the third party or service is unavailable in an ephemeral environment?
## Pitfalls
You’ve read about the benefits and challenges. However, before proceeding, it’s important to understand the pitfalls. Otherwise, we risk confusing developers, upsetting leadership, or burning through budget. Here are some common pitfalls we see:
- Cost: In the current climate, teams are actively cutting costs to offset ballooning cloud infra. Without proper automation, monitoring, and management, launching a new environment for each pull request could create massive cost overruns.
- Infrastructure Sharing: To address provisioning time and cost overruns, many teams rely on shared infrastructure. Without proper isolation of shared resources, managing infrastructure becomes complex and error-prone.
- Architecture Divergence: It’s fine for the architecture of an ephemeral environment to be different from production. If it’s too different, you diminish the value of ephemeral environments. Ultimately, we’re trying to simulate conditions as close to production.
## Conclusion
In conclusion, ephemeral environments present an innovative approach to the application development and deployment process. They offer unique benefits such as resolving staging bottlenecks and enhancing security and performance testing. However, they also bring challenges such as provisioning time and data seeding. Despite these hurdles, with the right strategies and tools, they can significantly improve the quality and speed of software development.
