Match Containers to Host Processes
@ Theo | Saturday, Feb 23, 2019 | 3 minutes read | Update at Saturday, Feb 23, 2019

During my presentation Securing Container Workloads on AWS Fargate, I built a demo environment where I could build and run various containers and show the effect they had on the host. While my demo went well, a key piece of feedback is that customers liked how I presented the demo environment by having containers and their host processes on one side. To that end, I’ll show you.

Containers Pane

To show the currently running containers on a given host, use docker ps. The normal format (for v18.09.1) looks like:

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
525a7b49ef67        nginx               "nginx -g 'daemon of…"   About an hour ago   Up About an hour    80/tcp                   tender_shirley

However, for this demo, I was only concerned with the name, image, command, and current status (which has the time it’s been running), so I formatted the output using the --format flag, and stuck it inside watch to update every second.

Command
watch -n 1 "docker ps --format 'table {{.Names}}\t{{.Image}}\t{{.Command}}\t{{.Status}}'"
Output
Every 1.0s: docker ps --format 'table {{.Names}}\t{{.Image}}\t{{.Command}}\t{{.Status}}'                            localhost.localdomain: Sat Feb 23 13:44:58 2019

NAMES               IMAGE               COMMAND                  STATUS
tender_shirley      nginx               "nginx -g 'daemon of  "   Up About an hour

Host Processes Pane

Getting the host processes (and a way to map them to containers) was more difficult. The best tool in Linux for looking at processes is ps (which is where Docker gets the name for docker ps), but this doesn’t give us all the information about a container.

When a container starts, it spawns as a process with a specific process identifier (PID) in the host, but the container sees the PID as 1. This process can also spawn other processes, which will reference ther parent process PPID. Subprocesses will show up with a PPID of the main process PID but inside the container as PPID 1. For my demo, I wanted to show both the processes and subprocesses at the host, and include information about the user running each process.

Thus, I built a script called watchpids.sh. This script gathered the host PIDs, found all of the children PIDs and then fed the list of PIDs to ps, also formatting the list to show the running time of the process, the PID, the PPID, the user associated with the process, and the command run. Again, execution of the script was wrapped in watch.

Script

With both the containers and processes displayed, map the container STATUS to the host process ELAPSED time to see what processes show up on the host whenever a new container is started.

Terminal Window

Tying it all together, I used tmux to build the container and host process panes on the right, and an area to type commands on the left.

tmux uses either keyboard shortcuts or commands inside the session to change the environment–going for a “scripting” approach, I chose the latter.

Commands
tmux new-session -d -s builder_demo
tmux split-window -h
tmux split-window -dv "watch -n 1 \"docker ps --format 'table {{.Names}}\t{{.Image}}\t{{.Command}}\t{{.Status}}'\""
tmux select-pane -t 0
tmux send-keys -t 1 'watch -n 1 ./watchpids.sh' C-m
tmux -2 attach-session -d
Screenshot

About Me

Self-described technology enthusiast working with containers, DevOps, networking, load balancing, etc.

Career

After college, I came back to the family business, this time to force-feed technology into the business instead of passing around a QuickBooks file and design templates on a Zip disk. This ended up as a good trade–I was able to both freely learn and implement new(er) technology and gain powerful business experience. I am fully capable of explaining any technical topic to a non-technical audience. I taught my mother about files/folders on a hard disk by showing her the files and folders in her file cabinets.

I spent a short time as a law firm doing more of the same, but wanted more. I joined a state-level government agency and began to specialize in networking. I quickly moved through the ranks moving from Junior to Senior status, and spent a few years as a Network Manager. I dove into “network service” technologies and tools like load balancing, name resolution, monitoring, logging, and analysis. My success there came from four principles:

  • Work with the customer–ensure your decisions are for their benefit.
  • The borders of your responsibility are soft–learn about how your department affects other departments. A little cross-team knowledge goes a long way.
  • Don’t waste time repeating processes–if you’ll repeat it, script it and let the system work for you.
  • Automate yourself out of a job–if you do, they’ll give you a better one.

College

I went to Florida Institute of Technology in Melbourne, FL, USA and received my Bachelor of Science in Computer Science. While the degree is a great résumé builder, the knowledge and experience gained were much more valuable.

We didn’t just focus on learning a programming language–we learned WHY a language was developed and what separates it from others. Concepts were more important, because that led to a language-independent programming skill. As a result, I can now write code in any language.

I also got a taste at other IT-related skills. The program provided enough electives for us to branch out and “test the waters” around different disciplines. As a result, I got a breadth of skills to help complement my degree: cryptography, computer vision, system administration, OS concepts, database design, etc.

My senior project was a collaboration between Aerospace, Mechanical, Computer, Electrical, and Software Engineers. We built a scale model of a V-22 “Osprey” with a design for mid-air transition while carrying heavy cargo. Since it was a scale model, we also used a wireless serial transmitter and ground interface to control the osprey using a Radio Controller hooked up to a computer screen. My job was the GUI/software for the Flight Control System and interface as well as the scripts to perform the advanced aeronautical calculations. It was a great team experience that further expanded my breath of skills and abilities.

Moving

My family and I wanted to move from Tallahassee, FL, USA, to Charlotte, NC, USA and we got the opportunity when I was offered a Network Engineer position with an insurance company that had a regional headquarters in Charlotte. I joined the Network Services team and found my passion for improving processes through orchestration/automation. I also got my first taste of cloud and cloud networking, which required a new education on cloud networking. For many years, I had watched other network professionals accelerate their knowledge and experience on networking to a point, then stick with that knowledge until otherwise forced to change. I realized that I’m not an “old school” networker, as I think being an expertise in networking doesn’t mean knowing every command in a CLI. Cloud networking is different, and requires a new way of architecting–traditional networking tools only work until the cloud border. Ultimately, I spent a short time at the insurance company because I was recruited by Amazon.

The Early Years

Ask my mother, and I was always going to work in technology. At age 5, I set the clock on the VCR and programmed it to record my shows.

My family owns a swimming pool contracting business in Tallahassee, FL, USA and I spent my childhood and teenage years learning how to run a business. Technology was a hobby, and I had fun exploring building my own gaming rig, writing plugins for software, and begrudgingly providing free technical support to friends and family.