Session 01¶

hey everyone! 👋

→ whoami
Johann Rudi, *he/him/his*
Department of Mathematics / College of Science / Virginia Tech
<https://personal.math.vt.edu/jrudi/>

my work

• since PhD: on large-scale parallel numerical solvers
• currently: inverse problems, machine learning & AI
• over 15 years of experience in parallel programming primarily in C with MPI/OpenMP

Today¶

1. introduction to the course
2. discussion about AI
3. terminal / command line interface

Why bother about scalable computing?¶

💬

grep -Er "scalable|large-scale" *

Relevance of scalable computing¶

• large problems → large-scale computing
• faster compute capabilities → getting to the solution faster
• scaling solution algorithms: \(10^0, 10^3, 10^6, 10^9,\) …
  … this becomes challenging very quickly

Why the emphasis on scaling?¶

The limits of physics require us to think about scaling.

Source: karlrupp/microprocessor-trend-data

transistor count grew exponentially

→ transistor density doubled every 18-24 months (Moore’s law)

major change in ~2005: frequency & power limits

→ more cores per processor

The flip side¶

Nature article published in September 2018

About the course¶

--help

Where you find everything¶

open https://code.vt.edu/jrudi/cmda3634

Course overview¶

ls *

over the semester, we’ll visit three topics …

1. Interfaces¶

Think of these as your command deck.

We’ll explore the tools that make human-computer interaction smooth, efficient, and even enjoyable:

• terminals,
• editors,
• version control,
• AI agents, and more.

2. The Go language and concurrency¶

Stretch your mind toward parallel thinking.

Learn to orchestrate tasks that run side-by-side, gaining a deep sense of how independent actions can cooperate. You’ll understand essential concepts:

• scalability
• shared memory among workers
• message passing between workers

3. Neural networks and PyTorch¶

Climb from lower-level mechanics to higher-level intelligence.

You’ll pull back the curtain on how neural networks are trained, appreciating the craft beneath Python’s and PyTorch’s polished interfaces.

You’ll be able to train using so-called distributed data parallelism.

Your role in the course¶

Readme at code.vt.edu/jrudi/cmda3634:

• What to know before you commit
• Do we have a textbook?
• What will you be working on?
• How you earn credit

AI¶

sudo

AI in this course¶

Readme at code.vt.edu/jrudi/cmda3634:

• Can you use AI?
• Use AI, but do it openly

Documenting AI use, see the Course Policy:

• AI Use and Disclosure

AI access¶

major companies with large language models (LLMs)

• Anthropic Claude
• Google Gemini (allows a free API key)
• Mistral Chat (allows a free API key)
• OpenAI GPT

VT ARC LLM services¶

Virginia Tech’s Advanced Research Computing (arc.vt.edu) hosts LLMs that are free to use for us.

→ ARC docs on AI

AI coding agents¶

AI via web 🐇 → AI via agents 🚀

• Gemini CLI (free & paid)
• GitHub Copilot in VSCode (free access via GitHub Education, use any LLM)
• Goose (open-source, use any LLM)
• OpenCode (open-source, use any LLM)
• Mistral Vibe (open-source, free & paid)
• Warp (entire terminal, free & paid)

New topic¶

cd

Topic 1: Interfaces¶

• terminal
• SSH
• Zsh
• editors
• Git
• AI agents

Terminal / command line interface¶

Terminal - why bother?¶

💬

Terminal - advantages¶

• keyboard efficiency
• reproducibility
• direct path to automation
• interaction used by AI agents

Using the terminal¶

wiki → Terminal

cat wiki/terminal.md

Demo¶

• essential commands: ls, cd, pwd, mv, cp, rm, mkdir, cat, less
• shortkeys: Tab, Up, Ctrl-r
• productivity commands: grep, find, rsync

Let’s look at an example of using terminal commands. We’re somewhat following the demonstration of: 50 Linux commands for beginners (by Dispatch)

• let’s look at this repository: ls
• wonder what’s inside code/data; cd into it; verify with pwd
• want to start a new feature: mkdir feat, rename mv feat feature
• copy files into new folder: cp, verify ls
• want to split csv file, use touch, one to many: rm
• cluttered screen: clear
• inspect csv: cat, less, more, head, tail, wc
• variable: content=$(cat file.csv), how variable: echo $content
• redirect output: >, >>
• create a backup: rsync
• compress all backup files: tar -czf archive.tar.gz *
• transfer to a server: scp, ssh

Key takeaways¶

• this course and your role in it
• take your keyboard on a spin
• terminal means productivity and reproducibility
• challenge yourself: how fast can you type? → typ.ing

Looking ahead¶

• connect to any machine anywhere on Earth
• your terminal – beautiful

Prepare for the next session 🌱¶

sessions/02-todo