The dominant cluster manager for Spark, Hadoop YARN, did not support Docker containers until recently (Hadoop 3.1 release), and even today the support for Docker remains “experimental and non-complete”. In the big data and Apache Spark scene, most applications still run directly on virtual machines without benefiting from containerization. The popularity of Kubernetes as the new standard for container orchestration and infrastructure management follows from the popularity of Docker.
The software engineering world has fully adopted Docker, and a lot of tools around Docker have changed the way we build and deploy software – testing, CI/CD, dependency management, versioning, monitoring, security. The benefits that come with using Docker containers are well known: they provide consistent and isolated environments so that applications can be deployed anywhere – locally, in dev / testing / prod environments, across all cloud providers, and on-premise – in a repeatable way.
0 kommentar(er)
