python - How to add mysqlclient to a Poetry environment

Question

I'm creating a project which needs to make a connection from Python running in a docker container to a MySQL database running in another container. Currently, my docker-compose file looks like this:

version: "3"

services:

  login:
    build:
      context: ./services/login
      dockerfile: docker/Dockerfile
    ports:
      - "80:80"
    # Need to remove this volume - this is only for dev work
    volumes:
      - ./services/login/app:/app
    # Need to remove this command - this is only for dev work
    command: /start-reload.sh

  db_users:
    image: mysql
    volumes:
      - ./data/mysql/users_data:/var/lib/mysql
      - ./databases/users:/docker-entrypoint-initdb.d/:ro
    restart: always
    ports:
      - 3306:3306
    # Remove 'expose' below for prod
    expose:
      - 3306
    environment:
      MYSQL_ROOT_PASSWORD: password
      MYSQL_DATABASE: users
      MYSQL_USER: user
      MYSQL_PASSWORD: password

And my Dockerfile for the login service looks like this:

# Note: this needs to be run from parent service directory

FROM tiangolo/uvicorn-gunicorn-fastapi:python3.8

# Install Poetry
RUN curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | POETRY_HOME=/opt/poetry python && 
    cd /usr/local/bin && 
    ln -s /opt/poetry/bin/poetry && 
    poetry config virtualenvs.create false

# Copy using poetry.lock* in case it doesn't exist yet
COPY ./app/pyproject.toml ./app/poetry.lock* /app/

RUN poetry install --no-root --no-dev

COPY ./app /app

I am trying to connect my login service to db_users, and want to make use of mysqlclient, but when I run poetry add mysqlclient, I get an error which includes the following lines:

    /bin/sh: mysql_config: command not found
    /bin/sh: mariadb_config: command not found
    /bin/sh: mysql_config: command not found
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/33/5yy7bny964bb0f3zggd1b4440000gn/T/pip-req-build-lak6lqu7/setup.py", line 15, in <module>
        metadata, options = get_config()
      File "/private/var/folders/33/5yy7bny964bb0f3zggd1b4440000gn/T/pip-req-build-lak6lqu7/setup_posix.py", line 70, in get_config
        libs = mysql_config("libs")
      File "/private/var/folders/33/5yy7bny964bb0f3zggd1b4440000gn/T/pip-req-build-lak6lqu7/setup_posix.py", line 31, in mysql_config
        raise OSError("{} not found".format(_mysql_config_path))
    OSError: mysql_config not found
    mysql_config --version
    mariadb_config --version
    mysql_config --libs
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

I'm assuming this is something to do with the fact that I need the mysql-connector-c library to work, but I'm not sure how to go about getting this in poetry.

I was looking at following this tutorial, but since I'm not running MySQL locally but rather in docker, I'm not sure how to translate those steps to work in docker.

So essentially, my question is two-fold:

1. How do I add mysqlclient to my pyproject.toml file
2. How do I get this working in my docker env?

question from:https://stackoverflow.com/questions/65923591/how-to-add-mysqlclient-to-a-poetry-environment

Answer 1

Much thanks to @sinoroc for your help! I was forgetting that my dev environment is also in Docker so I didn't really need to care about the poetry environment.

With that said, I edited the Dockerfile to look like the below:

# Note: this needs to be run from parent service directory

FROM tiangolo/uvicorn-gunicorn-fastapi:python3.8

RUN apt-get install default-libmysqlclient-dev

# Install Poetry
RUN curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | POETRY_HOME=/opt/poetry python && 
    cd /usr/local/bin && 
    ln -s /opt/poetry/bin/poetry && 
    poetry config virtualenvs.create false

# Copy using poetry.lock* in case it doesn't exist yet
COPY ./app/pyproject.toml ./app/poetry.lock* /app/

RUN poetry install --no-root --no-dev

COPY ./app /app

Which now has everything working as expected.