Skip to main content

Batch Clone & Pull Multiple Git Repos with a Bash Script

· 6 min read
Hieu Nguyen
Hieu Nguyen
Senior Software Engineer at OCB

Automate cloning, checking out, and pulling multiple Git repositories at once using simple bash scripts — plus auto-generating a Maven parent POM for Java projects.

When you join a new company or get assigned to a new feature, you often need to clone multiple repositories — the main service plus several shared libraries. Doing this manually means typing git clone, cd, git checkout, git pull over and over. It gets old fast.

I ran into this exact problem working on Java microservices in banking, where a single project depended on 5+ library repos. So I wrote a few bash scripts to automate the entire setup in one command. Here's what I learned.

Bash Basics (Quick Recap)​

If you're already comfortable with bash, skip to the Clone Script section.

Hello World​

Create a file called hello.sh:

#!/bin/bash
echo "Hello World!"

Make it executable and run:

chmod +x hello.sh
./hello.sh

Variables​

#!/bin/bash
name="Noka"
echo "Hello, $name!"  # Output: Hello, Noka!

Parameters​

Bash scripts accept positional arguments via $1, $2, etc.:

#!/bin/bash
echo "Hello, $1!"
./hello.sh Noka
# Output: Hello, Noka!

Conditionals​

if [ "$folder" = "library" ]; then
    echo "This is the library folder"
else
    echo "Not the library folder"
fi

Loops​

# Loop through all items in the current directory
for item in *; do
    echo "$item"
done

Clone Multiple Repos​

Here's the scenario: you need to clone a main project and several shared libraries into a structured folder:

project/
├── main-service/          ← main project
└── library/               ← shared libraries
    ├── common-utils/
    ├── auth-lib/
    ├── payment-lib/
    └── notification-lib/

clone-source.sh​

#!/bin/bash

# Colors for pretty output
GREEN='\033[1;32m'
BLUE='\033[1;34m'
NC='\033[0m' # No Color

LIB_DIR="library"

echo -e "${GREEN}=== Cloning source code ===${NC}"

# Clone main project
echo -e "${BLUE}[1/2] Cloning main service...${NC}"
git clone git@github.com:your-org/main-service.git

# Create library directory and clone shared libs
echo -e "${BLUE}[2/2] Cloning libraries...${NC}"
mkdir -p "$LIB_DIR"
cd "$LIB_DIR"

git clone git@github.com:your-org/common-utils.git
echo -e "${GREEN}  ✓ common-utils${NC}"

git clone git@github.com:your-org/auth-lib.git
echo -e "${GREEN}  ✓ auth-lib${NC}"

git clone git@github.com:your-org/payment-lib.git
echo -e "${GREEN}  ✓ payment-lib${NC}"

git clone git@github.com:your-org/notification-lib.git
echo -e "${GREEN}  ✓ notification-lib${NC}"

cd ..

echo -e "${GREEN}=== Clone complete! ===${NC}"

Usage:

chmod +x clone-source.sh
./clone-source.sh

Checkout All Repos to the Same Branch​

When you need all repos on the same feature branch (e.g., dev or feature/payment-v2):

checkout-source.sh​

#!/bin/bash

# Colors
RED='\033[1;31m'
GREEN='\033[1;32m'
BLUE='\033[1;34m'
NC='\033[0m'

BRANCH="$1"
LIB_DIR="library"

if [ -z "$BRANCH" ]; then
    echo -e "${RED}Usage: ./checkout-source.sh <branch-name>${NC}"
    exit 1
fi

echo -e "${BLUE}=== Checking out branch: ${BRANCH} ===${NC}"

for folder in */; do
    # Remove trailing slash
    folder="${folder%/}"

    if [ "$folder" = "$LIB_DIR" ]; then
        # Library folder — checkout each sub-folder
        echo -e "${GREEN}📁 ${folder}/${NC}"
        for sub_folder in "$folder"/*/; do
            sub_folder="${sub_folder%/}"
            sub_name=$(basename "$sub_folder")
            cd "$sub_folder"
            echo -e "  ${BLUE}→ ${sub_name}${NC}"
            git checkout "$BRANCH" 2>/dev/null || echo -e "  ${RED}  ⚠ Branch not found: ${BRANCH}${NC}"
            cd ../..
        done
    elif [ -d "$folder/.git" ]; then
        # Regular git repo
        echo -e "${GREEN}📁 ${folder}${NC}"
        cd "$folder"
        git checkout "$BRANCH" 2>/dev/null || echo -e "  ${RED}⚠ Branch not found: ${BRANCH}${NC}"
        cd ..
    fi
done

echo -e "${BLUE}=== Checkout complete ===${NC}"

Usage:

chmod +x checkout-source.sh
./checkout-source.sh dev
./checkout-source.sh feature/payment-v2

Pull All Repos at Once​

Same pattern — loop through all repos and pull the latest changes:

pull-source.sh​

#!/bin/bash

# Colors
GREEN='\033[1;32m'
BLUE='\033[1;34m'
RED='\033[1;31m'
NC='\033[0m'

LIB_DIR="library"

echo -e "${BLUE}=== Pulling latest changes ===${NC}"

for folder in */; do
    folder="${folder%/}"

    if [ "$folder" = "$LIB_DIR" ]; then
        echo -e "${GREEN}📁 ${folder}/${NC}"
        for sub_folder in "$folder"/*/; do
            sub_folder="${sub_folder%/}"
            sub_name=$(basename "$sub_folder")
            cd "$sub_folder"
            echo -e "  ${BLUE}→ ${sub_name}${NC}"
            git pull 2>/dev/null || echo -e "  ${RED}  ⚠ Pull failed${NC}"
            cd ../..
        done
    elif [ -d "$folder/.git" ]; then
        echo -e "${GREEN}📁 ${folder}${NC}"
        cd "$folder"
        git pull 2>/dev/null || echo -e "  ${RED}⚠ Pull failed${NC}"
        cd ..
    fi
done

echo -e "${BLUE}=== Pull complete ===${NC}"

Bonus: Auto-Generate Maven Parent POM​

If you're working with Java Maven projects, you can auto-generate a parent pom.xml that includes all cloned repos as modules. This lets you open everything in IntelliJ as a single project.

generate-pom.sh​

#!/bin/bash

GREEN='\033[1;32m'
BLUE='\033[1;34m'
NC='\033[0m'

LIB_DIR="library"
POM_FILE="pom.xml"

echo -e "${BLUE}=== Generating parent pom.xml ===${NC}"

# Write POM header
cat > "$POM_FILE" << 'EOF'
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
         http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.hoclamdev</groupId>
    <artifactId>workspace-parent</artifactId>
    <version>1.0.0</version>
    <packaging>pom</packaging>
    <modules>
EOF

# Add modules dynamically
ROOT_PATH="$(pwd)/"

for folder in */; do
    folder="${folder%/}"

    if [ "$folder" = "$LIB_DIR" ]; then
        for sub_folder in "$folder"/*/; do
            sub_folder="${sub_folder%/}"
            echo "        <module>${sub_folder}</module>" >> "$POM_FILE"
            echo -e "  ${GREEN}✓ ${sub_folder}${NC}"
        done
    elif [ -d "$folder/.git" ] && [ -f "$folder/pom.xml" ]; then
        echo "        <module>${folder}</module>" >> "$POM_FILE"
        echo -e "  ${GREEN}✓ ${folder}${NC}"
    fi
done

# Write POM footer
cat >> "$POM_FILE" << 'EOF'
    </modules>
    <properties>
        <maven.compiler.source>17</maven.compiler.source>
        <maven.compiler.target>17</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>
</project>
EOF

echo -e "${BLUE}=== Generated ${POM_FILE} ===${NC}"
echo ""
echo "Open IntelliJ → File → Open → select the generated pom.xml"

This produces a pom.xml like:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" ...>
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.hoclamdev</groupId>
    <artifactId>workspace-parent</artifactId>
    <version>1.0.0</version>
    <packaging>pom</packaging>
    <modules>
        <module>main-service</module>
        <module>library/common-utils</module>
        <module>library/auth-lib</module>
        <module>library/payment-lib</module>
        <module>library/notification-lib</module>
    </modules>
    <properties>
        <maven.compiler.source>17</maven.compiler.source>
        <maven.compiler.target>17</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>
</project>

Tip: Open this pom.xml in IntelliJ IDEA → it will import all modules as a single workspace, with cross-module navigation and dependency resolution.

Complete Workflow​

Here's the full workflow to set up a new project in one go:

# 1. Clone everything
./clone-source.sh

# 2. Checkout the right branch
./checkout-source.sh dev

# 3. Generate parent POM (Java projects)
./generate-pom.sh

# 4. Open in IntelliJ and start coding! 🎉

For daily development:

# Pull latest changes across all repos
./pull-source.sh

Key Takeaways​

  1. Automate repetitive Git workflows — if you're typing the same commands across 5+ repos, write a script.
  2. Use colored output — it makes scripts easier to debug and looks professional.
  3. Add error handling — check if branches exist before switching, handle pull failures gracefully.
  4. Generate parent POMs — for Java projects, a multi-module POM lets IntelliJ treat everything as one workspace.
  5. Keep scripts in the project root — commit them to Git so your team can use them too.

Thanks for reading! If you're working with multiple repos in your daily workflow, these scripts can save you a lot of time. Also check out my Ubuntu post-install guide and LazyVim + Tmux setup for more dev environment tips. 🚀