medusa/the_information_nexus
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
63cffe74009d51382291242d5682777ca4fd9487
the_information_nexus/tech_docs/docker_dev_enviroments.md

7.0 KiB
Raw Blame History

Certainly! Let’s take a step back and define more logical and well-defined environments tailored to your needs. We’ll create distinct Docker environments for JSON processing, API interactions, notebook visualizations, data analysis, web development, DevOps, and testing.

Logical Environments

  1. JSON Processing Environment:

    • Purpose: To handle JSON parsing, validation, and transformation.
    • Dependencies: Libraries for working with JSON data.

  2. API Interaction Environment:

    • Purpose: To interact with APIs, handle requests, and manage responses.
    • Dependencies: HTTP clients and frameworks for API interactions.

  3. Notebook Visualization Environment:

    • Purpose: To perform data analysis and visualization in Jupyter notebooks.
    • Dependencies: Libraries for data manipulation and visualization.

  4. General Data Analysis Environment:

    • Purpose: To analyze and process data using various data science tools.
    • Dependencies: Libraries for data manipulation, analysis, and machine learning.

  5. Web Development Environment:

    • Purpose: To develop web applications and APIs.
    • Dependencies: Web frameworks and tools for developing web services.

  6. DevOps Environment:

    • Purpose: To manage infrastructure, automate tasks, and handle CI/CD pipelines.
    • Dependencies: Tools for infrastructure as code, automation, and cloud services.

  7. Testing Environment:

    • Purpose: To run unit tests, integration tests, and other forms of automated testing.
    • Dependencies: Testing frameworks and tools for code coverage.

Refactored Environments

1. JSON Processing Environment

Dockerfile.json-processing:

FROM python:3.9

RUN apt-get update && \
    apt-get install -y vim git curl

RUN mkdir -p ~/.vim/pack/plugins/start && \
    curl -fLo ~/.vim/autoload/plug.vim --create-dirs \
    https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim

COPY .vimrc /root/.vimrc

WORKDIR /app

COPY requirements-json.txt /app/

RUN pip install --no-cache-dir -r requirements-json.txt

COPY . /app

CMD ["tail", "-f", "/dev/null"]

requirements-json.txt:

jsonschema
pydantic
ujson

2. API Interaction Environment

Dockerfile.api-interaction:

FROM python:3.9

RUN apt-get update && \
    apt-get install -y vim git curl

RUN mkdir -p ~/.vim/pack/plugins/start && \
    curl -fLo ~/.vim/autoload/plug.vim --create-dirs \
    https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim

COPY .vimrc /root/.vimrc

WORKDIR /app

COPY requirements-api.txt /app/

RUN pip install --no-cache-dir -r requirements-api.txt

COPY . /app

CMD ["tail", "-f", "/dev/null"]

requirements-api.txt:

requests
httpx
fastapi
pydantic
ujson

3. Notebook Visualization Environment

Dockerfile.notebook-visualization:

FROM python:3.9

RUN apt-get update && \
    apt-get install -y vim git curl

RUN mkdir -p ~/.vim/pack/plugins/start && \
    curl -fLo ~/.vim/autoload/plug.vim --create-dirs \
    https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim

COPY .vimrc /root/.vimrc

WORKDIR /app

COPY requirements-visualization.txt /app/

RUN pip install --no-cache-dir -r requirements-visualization.txt

COPY . /app

CMD ["jupyter", "notebook", "--ip=0.0.0.0", "--allow-root", "--no-browser"]

requirements-visualization.txt:

numpy
pandas
matplotlib
seaborn
jupyter
plotly

4. General Data Analysis Environment

Dockerfile.data-analysis:

FROM python:3.9

RUN apt-get update && \
    apt-get install -y vim git curl

RUN mkdir -p ~/.vim/pack/plugins/start && \
    curl -fLo ~/.vim/autoload/plug.vim --create-dirs \
    https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim

COPY .vimrc /root/.vimrc

WORKDIR /app

COPY requirements-data-analysis.txt /app/

RUN pip install --no-cache-dir -r requirements-data-analysis.txt

COPY . /app

CMD ["tail", "-f", "/dev/null"]

requirements-data-analysis.txt:

numpy
pandas
scipy
scikit-learn
matplotlib
seaborn
jupyter

5. Web Development Environment

Dockerfile.web-development:

FROM python:3.9

RUN apt-get update && \
    apt-get install -y vim git curl

RUN mkdir -p ~/.vim/pack/plugins/start && \
    curl -fLo ~/.vim/autoload/plug.vim --create-dirs \
    https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim

COPY .vimrc /root/.vimrc

WORKDIR /app

COPY requirements-web-dev.txt /app/

RUN pip install --no-cache-dir -r requirements-web-dev.txt

COPY . /app

CMD ["tail", "-f", "/dev/null"]

requirements-web-dev.txt:

flask
django
fastapi
requests
httpx

6. DevOps Environment

Dockerfile.devops:

FROM python:3.9

RUN apt-get update && \
    apt-get install -y vim git curl

RUN mkdir -p ~/.vim/pack/plugins/start && \
    curl -fLo ~/.vim/autoload/plug.vim --create-dirs \
    https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim

COPY .vimrc /root/.vimrc

WORKDIR /app

COPY requirements-devops.txt /app/

RUN pip install --no-cache-dir -r requirements-devops.txt

COPY . /app

CMD ["tail", "-f", "/dev/null"]

requirements-devops.txt:

ansible
terraform
docker
boto3

7. Testing Environment

Dockerfile.testing:

FROM python:3.9

RUN apt-get update && \
    apt-get install -y vim git curl

RUN mkdir -p ~/.vim/pack/plugins/start && \
    curl -fLo ~/.vim/autoload/plug.vim --create-dirs \
    https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim

COPY .vimrc /root/.vimrc

WORKDIR /app

COPY requirements-testing.txt /app/

RUN pip install --no-cache-dir -r requirements-testing.txt

COPY . /app

CMD ["tail", "-f", "/dev/null"]

requirements-testing.txt:

pytest
mock
coverage
tox

Managing Multiple Dockerfiles

You can use descriptive names for the Dockerfiles and build commands specifying the Dockerfile to use.

Building Containers:

docker build -t json-processing-env -f Dockerfile.json-processing .
docker build -t api-interaction-env -f Dockerfile.api-interaction .
docker build -t notebook-visualization-env -f Dockerfile.notebook-visualization .
docker build -t data-analysis-env -f Dockerfile.data-analysis .
docker build -t web-development-env -f Dockerfile.web-development .
docker build -t devops-env -f Dockerfile.devops .
docker build -t testing-env -f Dockerfile.testing .

Running Containers:

docker run -it --name json-processing-container json-processing-env
docker run -it --name api-interaction-container api-interaction-env
docker run -it --name notebook-visualization-container -p 8888:8888 notebook-visualization-env
docker run -it --name data-analysis-container data-analysis-env
docker run -it --name web-development-container web-development-env
docker run -it --name devops-container devops-env
docker run -it --name testing-container testing-env

This structure ensures each environment is well-defined, logical, and tailored to specific tasks, allowing you to work efficiently across different projects and avoid conflicts between dependencies.