Container Filesystems
About
A container filesystem is a combination of image layers and a runtime writable layer. It provides a full Linux file system experience inside a containerized environment, isolated from the host and from other containers.
The filesystem seen inside a container is:
Composed of layers defined by the container image
Immutable in its base layers
Overlaid with a writable layer at runtime for file modifications
Ephemeral, meaning changes do not persist across container restarts unless backed by external storage
Layered Image Filesystem Internals
Each container image consists of multiple read-only layers. These are stacked in order and form the base of the filesystem. When a container runs:
The container runtime (e.g., CRI-O in OpenShift) creates a new top writable layer
All file operations inside the container interact with this top writable layer
If a process reads a file, it fetches it from the topmost layer where it exists
If modified, it is copied from the lower layer into the writable layer (copy-on-write)
If created, it exists only in the writable layer
If deleted, it is masked in the writable layer
This filesystem stack is managed by a union or overlay filesystem (like overlayfs), abstracted by the container runtime.
Lifecycle and Persistence Characteristics
Because the writable layer is created at container start and destroyed at container exit, the container’s default filesystem has these properties:
Non-persistent: Any files written to default paths like
/,/tmp, or/varare lost after the pod is deleted or restartedNon-shared: Files written inside one container are not accessible to another container (even within the same pod), unless explicitly shared via volumes
Isolated: Container filesystem has no direct visibility into the host file system
This design enforces statelessness in containers and mandates use of volumes for persistence.
Filesystem Layout: What Exists Inside a Container
Depending on the base image (Ubuntu, Alpine, OpenJDK, etc.), we’ll typically find:
/: Root directory/tmp: Temporary files/var: Application data, logs/etc: Configuration files (read-only)/usr: Binaries and libraries/home: User home directory (optional)/proc,/sys,/dev: Pseudo-filesystems mounted by the runtime
Everything outside mounted volumes is within the ephemeral container layer stack.
The Role of /tmp
/tmpThe /tmp directory is almost universally present and writable. It is used by:
Java and Spring Boot for temporary files
Compilers, installers, and CLI tools
Temporary downloads, caches, and extracted content
Spring Boot, by default, writes uploaded multipart files to /tmp.
However:
It exists only in the container's ephemeral layer
Once the pod restarts, the contents of
/tmpare lostIt should not be used to store anything that must survive restarts or be accessed across replicas
Filesystem with Volumes: Overlay Behavior
When a volume is mounted into a container in OpenShift or Kubernetes, it interacts with the container’s existing filesystem in a very specific way. This behavior is important to understand because it affects what the application can see and access at the mount path.
By default, every container has a complete Linux-like filesystem derived from the image it is built from. This includes standard directories such as /app, /etc, /tmp, /usr, and so on.
However, when we mount a volume into a specific path inside the container (for example, mounting a PersistentVolumeClaim to /data), the system performs what is known as a bind mount. This action overrides whatever was present in that path inside the original container image.
This means:
The mounted volume becomes the new content of that directory inside the container.
The original files or folders at that mount path (as defined in the image) are hidden or masked.
Only the data present in the volume is now visible to the application.
This behavior is part of how overlay or union filesystems work in container runtimes. The volume effectively replaces that section of the image's filesystem during runtime.
Suppose our container image has a directory /config that contains some pre-built default configuration files. We then mount a ConfigMap to /config.
Now:
The original files in
/configthat were part of the image are no longer visible.Only the files from the ConfigMap are available in
/configinside the running container.
If the ConfigMap contains fewer files than what was in the image, the missing files from the image will not be accessible anymore.
This behavior is important to keep in mind because:
If we accidentally mount a volume to a path that already contains important files from the image, we will lose access to those files during runtime.
If our application depends on pre-existing configuration files, scripts, or binaries at a path where we mount a volume, those resources will be hidden unless we ensure the volume includes everything needed.
Mounting over sensitive paths like
/etc,/usr, or/libcan cause containers to fail at runtime due to missing system files or broken dependencies.
Filesystem Security Context and Permissions
When we run containers in OpenShift, they are not allowed to run as root by default. This means the filesystem inside the container is subject to specific user, group, and permission rules, which can affect whether the container can read from or write to certain directories.
This is controlled through the Security Context and the filesystem permissions applied to mounted volumes and internal paths.
Why OpenShift Does Not Allow Root Containers by Default
OpenShift follows Security-Enhanced Linux (SELinux) and other policies to isolate workloads and prevent privilege escalation. As a result:
Containers are assigned random, non-root UIDs at runtime.
The container runs as a user that is not known in the container’s
/etc/passwdfile.The container cannot access files or directories that require root privileges unless explicitly permitted.
This is done to improve isolation and reduce risk.
What Problems Can Arise
Because the container runs as a random UID, it might face permission issues when trying to:
Write to a directory that expects a specific user (e.g., owned by
root)Access mounted volumes that are not readable or writable by the container’s UID
Use directories with restrictive permissions (e.g.,
/var,/data,/logs)
For example:
A Spring Boot app that writes to
/data/uploadswill fail if that directory is not writable by the UID the container is using.A PVC mounted to
/cachemight be inaccessible if it is owned byroot:rootand does not grant group or world write permissions.
Understanding File Ownership Inside Containers
Linux files and directories have:
A user owner
A group owner
A set of permissions: read, write, execute for user, group, and others
When a container tries to write to a path:
The process checks if the current UID has write permission
If the container's UID does not match the file or directory’s owner, and group/others also lack write permissions, the write will fail
Security Context in OpenShift Pod Specifications
OpenShift allows we to define a security context in our pod/deployment YAML to control how the container interacts with the filesystem.
Key fields:
runAsUser: Specify a fixed UID instead of a random onefsGroup: Assigns a group ID to the container and mounted volumesAll mounted volumes will be owned by this group
The container gains group-level access
runAsNonRoot: Ensures container does not run as UID 0readOnlyRootFilesystem: Makes the entire container filesystem read-only except for explicitly writable volumes
fsGroup: How It Solves Write Access Issues
If we mount a PVC and the container cannot write to it, setting fsGroup helps.
Example:
This makes all mounted volumes accessible to group 1000, and the container process is added to this group.
This is especially useful when:
The storage backend (like NFS) does not support dynamic UID mapping
Our app image does not run as root
We need a consistent way to grant write access to volume
Pre-configuring Permissions with Init Containers
If fine-grained control is needed, we can use an init container to prepare the volume before the main app starts.
The init container can:
Run as root (if allowed)
Create directories
Set correct ownership (
chown) and permissions (chmod)
This prepares the mounted volume so that the application container (running as non-root) can write to it.
Last updated