Error response from daemon: unknown or invalid runtime name: nvidia

Troubleshooting and help
, ,
1

Hello,

I face with issue

“Error response from daemon: unknown or invalid runtime name: nvidia”

while trying deploy my service which uses GPU:

Dockerfile.koyeb

FROM koyeb/docker-compose

# Install Git

RUN apk add --no-cache git

# Copy application files

COPY . /app

Dockerfile:

# Change CUDA and cuDNN version here
FROM nvidia/cuda:12.4.1-cudnn-runtime-ubuntu20.04
ARG PYTHON_VERSION=3.10

ENV DEBIAN_FRONTEND=noninteractive
ENV PYTHONPATH=/app
ENV DOCKER_ENV=true
RUN apt-get update && apt-get install -y --no-install-recommends \
        software-properties-common \
        wget \
    && add-apt-repository ppa:deadsnakes/ppa \
    && apt-get update && apt-get install -y --no-install-recommends \
        python$PYTHON_VERSION \
        python$PYTHON_VERSION-dev \
        python$PYTHON_VERSION-venv \
    && wget https://bootstrap.pypa.io/get-pip.py -O get-pip.py \
    && python$PYTHON_VERSION get-pip.py \
    && rm get-pip.py \
    && ln -sf /usr/bin/python$PYTHON_VERSION /usr/bin/python \
    && ln -sf /usr/local/bin/pip$PYTHON_VERSION /usr/local/bin/pip \
    && python --version \
    && pip --version \
    && apt-get purge -y --auto-remove software-properties-common \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

####### Add your own installation commands here #######
# RUN pip install some-package
# RUN wget https://path/to/some/data/or/weights
# RUN apt-get update && apt-get install -y <package-name>

WORKDIR /app
COPY . /app

# Install litserve and requirements
RUN pip install --no-cache-dir -r requirements.txt
EXPOSE 8000

# Run FastAPI application
CMD ["uvicorn", "api.main:app", "--host", "0.0.0.0", "--port", "8000"]

docker-compose.yml

services:
  api:
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "8000:8000"
    environment:
      QDRANT_URL: ${QDRANT_URL:?Please set QDRANT_URL}
      QDRANT_API_KEY: ${QDRANT_API_KEY:?Please set QDRANT_API_KEY}
      QDRANT_TIMEOUT: ${QDRANT_TIMEOUT:-3}
      NVIDIA_VISIBLE_DEVICES: all
    volumes:
      - .:/app
    runtime: nvidia
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    restart: unless-stopped
    command: uvicorn api.main:app --host 0.0.0.0 --port 8000

I can build and run container on my local without any problem but can’t deploy to Koyeb

UPDATE:
Even though I removed runtime: nvidia, I get the following error:

Error response from daemon: could not select device driver “” with capabilities: [[gpu]]

docker-compose.yml

services:
  api:
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "8000:8000"
    environment:
      QDRANT_URL: ${QDRANT_URL:?Please set QDRANT_URL}
      QDRANT_API_KEY: ${QDRANT_API_KEY:?Please set QDRANT_API_KEY}
      QDRANT_TIMEOUT: ${QDRANT_TIMEOUT:-3}
    volumes:
      - .:/app
    deploy:
      resources:
        reservations:
          devices:
            - capabilities: [gpu]
    restart: unless-stopped
    command: uvicorn api.main:app --host 0.0.0.0 --port 8000
2

Hi @Gurur_Aser,

Do you have a service ID to provide so we could look into it?