mirror of
https://github.com/ROCm/composable_kernel.git
synced 2026-05-11 17:00:18 +00:00
b2ea5fd315
[CK] Fix CI Failures for PR From Forks MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Motivation Fork PRs fail CI when `RUN_AITER_TESTS` or `RUN_FA_TESTS` is enabled. The docker scripts run `git clone -b "$CK_*_BRANCH" https://github.com/ROCm/rocm-libraries.git`, but a fork's branch doesn't exist upstream: ``` fatal: Remote branch <fork-branch> not found in upstream origin ``` Example: [PR #6529 build #4](http://micimaster.amd.com/blue/organizations/jenkins/rocm-libraries-folder%2FComposable%20Kernel/detail/PR-6529/4/pipeline). ## Technical Details **`Jenkinsfile`** — for PRs, use the upstream-visible PR ref instead of the head branch name: ```groovy CURRENT_BRANCH_NAME = env.CHANGE_ID ? "refs/pull/${env.CHANGE_ID}/head" : (env.CHANGE_BRANCH ? env.CHANGE_BRANCH : env.BRANCH_NAME) ``` **`Dockerfile.aiter` / `Dockerfile.fa`** — `git clone -b <ref>` only accepts branches (`refs/heads/*`) and tags (`refs/tags/*`), so it can't resolve `refs/pull/N/head`. Switch to `git fetch`, which accepts any refspec (and still works for plain branch names): ```sh mkdir rocm-libraries && cd rocm-libraries git init -q git remote add origin https://github.com/ROCm/rocm-libraries.git git fetch --depth 1 --filter=blob:none origin "$CK_*_BRANCH" git sparse-checkout init --cone git sparse-checkout set projects/composablekernel git checkout FETCH_HEAD ``` `git checkout FETCH_HEAD` lands in detached HEAD, which breaks the existing `git branch -m "$CK_*_BRANCH"` (and that name isn't a valid local branch anyway). Decouple the local branch name from the upstream ref: - Replace `git init` + `git branch -m` with `git init -b "$LOCAL_BRANCH"` (requires git ≥ 2.28, satisfied by base images) - `LOCAL_BRANCH="ck-import-${ROCM_LIBRARIES_SHA}"` in the rocm-libraries path; `LOCAL_BRANCH="$CK_*_BRANCH"` in the fallback - Downstream `git clone -b ... ../ck` uses `$LOCAL_BRANCH` ## Test Plan Manually trigger a build on this PR with `RUN_AITER_TESTS=true` and `RUN_FA_TESTS=true`; both docker images should build end-to-end. ## Test Result [jenkins / rocm-libraries-folder/Composable Kernel / PR-6701 / #3](http://micimaster.amd.com/blue/organizations/jenkins/rocm-libraries-folder%2FComposable%20Kernel/detail/PR-6701/3/pipeline/) ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.
50 lines
2.4 KiB
Docker
50 lines
2.4 KiB
Docker
ARG BASE_DOCKER="rocm/pytorch:latest"
|
|
FROM $BASE_DOCKER
|
|
ARG FA_ORIGIN="ROCm"
|
|
ARG FA_BRANCH="tridao"
|
|
ARG CK_FA_ORIGIN="ROCm"
|
|
ARG CK_FA_BRANCH="develop"
|
|
# CK_FROM_ROCM_LIBRARIES - 1: CK from rocm-libraries sparse-checkout; 0: direct clone from ROCm/composable_kernel
|
|
ARG CK_FROM_ROCM_LIBRARIES=1
|
|
ARG GPU_ARCHS="gfx90a;gfx942;gfx950"
|
|
RUN set -x ; \
|
|
sudo mkdir /home/jenkins && \
|
|
sudo mkdir /home/jenkins/workspace && \
|
|
cd /home/jenkins/workspace && rm -rf rocm-libraries ck && \
|
|
if [ "$CK_FROM_ROCM_LIBRARIES" = "1" ]; then \
|
|
mkdir rocm-libraries && cd rocm-libraries && \
|
|
git init -q && \
|
|
git remote add origin https://github.com/$CK_FA_ORIGIN/rocm-libraries.git && \
|
|
git fetch --depth 1 --filter=blob:none origin "$CK_FA_BRANCH" && \
|
|
git sparse-checkout init --cone && \
|
|
git sparse-checkout set projects/composablekernel && \
|
|
git checkout FETCH_HEAD && \
|
|
ROCM_LIBRARIES_SHA=$(git rev-parse --short HEAD) && \
|
|
LOCAL_BRANCH="ck-import-${ROCM_LIBRARIES_SHA}" && \
|
|
mv projects/composablekernel ../ck && \
|
|
cd ../ck && rm -rf ../rocm-libraries && \
|
|
git init -b "$LOCAL_BRANCH" && \
|
|
git config user.name "assistant-librarian[bot]" && \
|
|
git config user.email "assistant-librarian[bot]@users.noreply.github.com" && \
|
|
git add -A && \
|
|
git commit -m "import from ROCm/rocm-libraries@$ROCM_LIBRARIES_SHA" > /dev/null ; \
|
|
else \
|
|
git clone --depth 1 -b "$CK_FA_BRANCH" https://github.com/$CK_FA_ORIGIN/composable_kernel.git ck && \
|
|
LOCAL_BRANCH="$CK_FA_BRANCH" ; \
|
|
fi && \
|
|
cd /home/jenkins/workspace && rm -rf flash-attention && \
|
|
git clone --depth 1 -b "$FA_BRANCH" --recursive "https://github.com/$FA_ORIGIN/flash-attention.git" && \
|
|
cd flash-attention && \
|
|
rm -rf csrc/composable_kernel/ && \
|
|
git clone -b "$LOCAL_BRANCH" ../ck csrc/composable_kernel/ && git add csrc/composable_kernel && \
|
|
MAX_JOBS=$(nproc) GPU_ARCHS="$GPU_ARCHS" /opt/venv/bin/python3 -u -m pip install --no-build-isolation -v . && \
|
|
groupadd -g 1001 jenkins && \
|
|
useradd -u 1001 -g 1001 -m -s /bin/bash jenkins && \
|
|
groupadd -f video && \
|
|
groupadd -f render && \
|
|
chown -R jenkins:jenkins /home/jenkins && \
|
|
chmod -R a+rwx /home/jenkins && \
|
|
chown -R jenkins:jenkins /tmp && \
|
|
chmod -R a+rwx /tmp && \
|
|
sudo usermod -aG irc jenkins
|