structure updates
This commit is contained in:
151
docs/tech_docs/python/Python Cheat Sheet.md
Normal file
151
docs/tech_docs/python/Python Cheat Sheet.md
Normal file
@@ -0,0 +1,151 @@
|
||||
# Python Cheat Sheet
|
||||
|
||||
## 1. Variables, Data Types, and Basic Operations
|
||||
|
||||
Python has several fundamental data types, including integers (int), floating point numbers (float), and strings (str). Python is a dynamically typed language, which means you don't need to declare the data type of a variable when you define it.
|
||||
|
||||
```python
|
||||
a = 10 # Integer
|
||||
b = 3.14 # Float
|
||||
c = "Hello, World!" # String
|
||||
```
|
||||
|
||||
Operators allow you to perform operations on variables. Arithmetic, comparison, assignment, logical, and identity operators are some of the main types in Python.
|
||||
|
||||
```python
|
||||
a = 10
|
||||
b = 20
|
||||
sum = a + b # Addition
|
||||
difference = a - b # Subtraction
|
||||
#... remaining code ...
|
||||
```
|
||||
|
||||
## 2. Control Structures (Conditionals and Loops)
|
||||
|
||||
Python uses `if`, `elif`, and `else` for conditional statements. Loops in Python can be programmed using a `for` or `while` loop.
|
||||
|
||||
```python
|
||||
# If-else statement
|
||||
if a > b:
|
||||
print("a is greater than b")
|
||||
else:
|
||||
print("a is not greater than b")
|
||||
|
||||
# For loop
|
||||
for i in range(5):
|
||||
print(i)
|
||||
```
|
||||
|
||||
## 3. Functions
|
||||
|
||||
Functions in Python are defined using the `def` keyword. They are used to encapsulate a piece of code that performs a specific task.
|
||||
|
||||
```python
|
||||
def greet(name):
|
||||
print("Hello, " + name)
|
||||
|
||||
greet("Alice")
|
||||
```
|
||||
|
||||
## 4. Lists, Tuples, Sets, and Dictionaries
|
||||
|
||||
Python has several types of compound data structures that can hold multiple values, including lists, tuples, sets, and dictionaries.
|
||||
|
||||
```python
|
||||
# Lists
|
||||
my_list = [1, 2, 3, 4, 5]
|
||||
|
||||
# Dictionaries
|
||||
my_dict = {"apple": 1, "banana": 2, "cherry": 3}
|
||||
```
|
||||
|
||||
## 5. File Handling
|
||||
|
||||
Python has built-in functions for reading and writing files. `open()` function is used to open a file. It returns a file object and is most commonly used with two arguments: `open(filename, mode)`.
|
||||
|
||||
```python
|
||||
# Reading a file
|
||||
file = open('file.txt', 'r')
|
||||
content = file.read()
|
||||
file.close()
|
||||
|
||||
# Writing to a file
|
||||
file = open('file.txt', 'w')
|
||||
file.write("Hello, World!")
|
||||
file.close()
|
||||
```
|
||||
|
||||
## 6. Exception Handling
|
||||
|
||||
Python uses `try` and `except` blocks to catch and handle exceptions. Python executes code following the `try` statement as a "normal" part of the program. The code that follows the `except` statement is the program's response to any exceptions in the preceding try clause.
|
||||
|
||||
```python
|
||||
try:
|
||||
# Code that may raise an exception
|
||||
x = 1 / 0
|
||||
except ZeroDivisionError:
|
||||
print("You can't divide by zero!")
|
||||
```
|
||||
|
||||
## 7. Classes and Objects
|
||||
|
||||
Python is an object-oriented programming language. You define classes to create your own types.
|
||||
|
||||
```python
|
||||
class MyClass:
|
||||
x = 5
|
||||
|
||||
p1 = MyClass() # Create an object of MyClass
|
||||
print(p1.x) # Access the properties of MyClass using the object
|
||||
```
|
||||
|
||||
## 8. Modules and Packages
|
||||
|
||||
Modules in Python are simply Python files with a .py extension, which implement a set of functions. A package is a way of organizing related modules into a single directory hierarchy.
|
||||
|
||||
```python
|
||||
# Importing a module
|
||||
import math
|
||||
print(math.sqrt(16)) # Use a function from the math module
|
||||
|
||||
# Importing
|
||||
ChatGPT
|
||||
a package
|
||||
import os.path
|
||||
print(os.path.join('/a', 'b', 'c')) # Use a function from the os.path package
|
||||
```
|
||||
|
||||
## 9. Working with JSON
|
||||
|
||||
Python has a built-in package called `json`, which can be used to work with JSON data.
|
||||
|
||||
```python
|
||||
import json
|
||||
|
||||
# Some JSON:
|
||||
x = '{"name": "John", "age": 30, "city": "New York"}'
|
||||
|
||||
# Parse x:
|
||||
y = json.loads(x)
|
||||
|
||||
# The result is a Python dictionary:
|
||||
print(y["age"])
|
||||
```
|
||||
|
||||
## 10. Making HTTP requests
|
||||
|
||||
The `requests` library is the de facto standard for making HTTP requests in Python. It abstracts the complexities of making requests behind a beautiful, simple API so that you can focus on interacting with services and consuming data in your application.
|
||||
|
||||
```python
|
||||
import requests
|
||||
|
||||
response = requests.get('https://www.example.com')
|
||||
|
||||
# Print the status code
|
||||
print(response.status_code)
|
||||
|
||||
# Print the content
|
||||
print(response.text)
|
||||
```
|
||||
|
||||
These are the basics to get you started with Python! Each of these topics has more depth to explore as you become more comfortable with the language.
|
||||
630
docs/tech_docs/python/Python-Virtual-Environments.md
Normal file
630
docs/tech_docs/python/Python-Virtual-Environments.md
Normal file
@@ -0,0 +1,630 @@
|
||||
# Python Virtual Environment Setup Guide
|
||||
|
||||
This guide provides step-by-step instructions on setting up a Python virtual environment and managing dependencies for your project.
|
||||
|
||||
## Creating a Virtual Environment
|
||||
|
||||
1. **Navigate to Your Project Directory**
|
||||
Open a terminal and navigate to the directory where your project is located.
|
||||
```bash
|
||||
cd path/to/your/project
|
||||
```
|
||||
|
||||
2. **Create the Virtual Environment**
|
||||
Use the `venv` module to create a virtual environment named `env`.
|
||||
```bash
|
||||
python3 -m venv env
|
||||
```
|
||||
|
||||
## Activating the Virtual Environment
|
||||
|
||||
1. **Activate the Environment**
|
||||
Once the environment is created, you need to activate it.
|
||||
```bash
|
||||
source env/bin/activate
|
||||
```
|
||||
After activation, your prompt will change to indicate that you are in the virtual environment.
|
||||
|
||||
## Managing Dependencies
|
||||
|
||||
1. **Create a Requirements File**
|
||||
Create a `requirements.txt` file to keep track of project dependencies.
|
||||
```bash
|
||||
touch requirements.txt
|
||||
```
|
||||
|
||||
2. **Freeze Installed Packages**
|
||||
If you have already installed packages, you can list them in `requirements.txt`.
|
||||
```bash
|
||||
pip freeze > requirements.txt
|
||||
```
|
||||
|
||||
3. **View the Requirements File**
|
||||
To see the contents of `requirements.txt`, use the `cat` command.
|
||||
```bash
|
||||
cat requirements.txt
|
||||
```
|
||||
|
||||
## Deactivating the Virtual Environment
|
||||
|
||||
1. **Deactivate the Environment**
|
||||
When you are done working in the virtual environment, deactivate it.
|
||||
```bash
|
||||
deactivate
|
||||
```
|
||||
|
||||
## Tips
|
||||
|
||||
- Always activate your virtual environment before working on your project.
|
||||
- Use `pip` to install any new packages while the environment is active.
|
||||
- Regularly update your `requirements.txt` file to reflect new dependencies.
|
||||
- Remember to deactivate the virtual environment when you're finished.
|
||||
|
||||
By following these steps, you can effectively manage your Python project's dependencies in a clean, isolated environment.
|
||||
|
||||
---
|
||||
## Standard Directory Structure for Python Projects on Debian
|
||||
|
||||
In a Python project, particularly on a Debian Linux system, it's important to follow a standard directory structure for organization and efficiency. Here is a recommended structure:
|
||||
|
||||
1. **src or app**: This directory holds the source code of the project.
|
||||
```bash
|
||||
mkdir src
|
||||
```
|
||||
|
||||
2. **docs**: This is where all the documentation related to the project should be kept.
|
||||
```bash
|
||||
mkdir docs
|
||||
```
|
||||
|
||||
3. **tests**: This directory will contain test scripts and test data.
|
||||
```bash
|
||||
mkdir tests
|
||||
```
|
||||
|
||||
4. **data**: If your project requires any data files, store them here. It's useful for projects that need to access datasets or other resources.
|
||||
```bash
|
||||
mkdir data
|
||||
```
|
||||
|
||||
5. **env**: This is for the Python virtual environment. Although it's common to place it in the project root, some prefer to keep it outside to prevent its accidental inclusion in version control systems like Git.
|
||||
- To create a virtual environment in Debian, navigate to your project root and run:
|
||||
```bash
|
||||
python3 -m venv env
|
||||
```
|
||||
- Remember to add `env/` to your `.gitignore` file if you decide to place it within the project root.
|
||||
|
||||
### Activating the Virtual Environment on Debian
|
||||
|
||||
- After creating the virtual environment, you can activate it using:
|
||||
```bash
|
||||
source env/bin/activate
|
||||
```
|
||||
|
||||
### Deactivating the Virtual Environment
|
||||
|
||||
- To exit the environment, simply run:
|
||||
```bash
|
||||
deactivate
|
||||
```
|
||||
|
||||
### Tips for Debian Users
|
||||
|
||||
- Always activate your virtual environment before starting work on the project.
|
||||
- Install project-specific Python packages within the virtual environment to avoid conflicts with system-wide packages.
|
||||
- Regularly update your `requirements.txt` file to keep track of dependencies.
|
||||
- Use relative paths in your scripts and tools for better portability and flexibility.
|
||||
|
||||
By following this structure and these tips, you can maintain a well-organized and efficient development environment for your Python projects on Debian.
|
||||
|
||||
---
|
||||
# Setting Up a Virtual Environment for KnowledgeBase-Tech Hugo Site
|
||||
|
||||
## Update PATH Environment Variable
|
||||
|
||||
Before installing `virtualenv`, ensure your PATH includes the directory where Python packages are installed. If you see a message like "The script virtualenv is installed in '/home/medusa/.local/bin' which is not on PATH," you'll need to update your PATH. Add the following line to your `.bashrc` or `.profile` file:
|
||||
|
||||
```bash
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
```
|
||||
|
||||
Then, reload your profile:
|
||||
|
||||
```bash
|
||||
source ~/.bashrc
|
||||
```
|
||||
|
||||
or for a login shell:
|
||||
|
||||
```bash
|
||||
source ~/.profile
|
||||
```
|
||||
|
||||
## Install Virtual Environment
|
||||
|
||||
Ensure Python and pip are installed, then install `virtualenv`:
|
||||
|
||||
```bash
|
||||
pip3 install virtualenv
|
||||
```
|
||||
|
||||
## Create the Virtual Environment
|
||||
|
||||
Create a virtual environment named `KnowledgeBase-Tech_env`:
|
||||
|
||||
```bash
|
||||
virtualenv KnowledgeBase-Tech_env
|
||||
```
|
||||
|
||||
## Activate the Virtual Environment
|
||||
|
||||
To start using the virtual environment:
|
||||
|
||||
```bash
|
||||
source KnowledgeBase-Tech_env/bin/activate
|
||||
```
|
||||
|
||||
## Install Dependencies
|
||||
|
||||
While the environment is active, install any required packages:
|
||||
|
||||
```bash
|
||||
pip install <package-name>
|
||||
```
|
||||
|
||||
## Freeze Requirements
|
||||
|
||||
To keep track of your project's dependencies, you can freeze the installed packages into a `requirements.txt` file:
|
||||
|
||||
```bash
|
||||
pip freeze > requirements.txt
|
||||
```
|
||||
|
||||
This file can be used to install the exact versions of the required packages on other setups or by other developers working on the project.
|
||||
|
||||
## Deactivate the Virtual Environment
|
||||
|
||||
When you're done working, deactivate the environment:
|
||||
|
||||
```bash
|
||||
deactivate
|
||||
```
|
||||
|
||||
## Working with the Environment
|
||||
|
||||
Remember to activate the `KnowledgeBase-Tech_env` environment every time you work on the KnowledgeBase-Tech Hugo site. This ensures all dependencies are isolated to this specific project.
|
||||
````
|
||||
|
||||
Replace `<package-name>` with the specific packages you need for your Hugo site. The addition of the freeze requirements section will help maintain a consistent set of dependencies across different development environments.
|
||||
|
||||
---
|
||||
|
||||
# Setting Up a Virtual Environment for KnowledgeBase-Tech Hugo Site
|
||||
|
||||
## Install Virtual Environment
|
||||
|
||||
First, ensure Python and pip are installed, then install `virtualenv`:
|
||||
|
||||
```bash
|
||||
pip3 install virtualenv
|
||||
```
|
||||
|
||||
## Create the Virtual Environment
|
||||
|
||||
Create a virtual environment named `KnowledgeBase-Tech_env`:
|
||||
|
||||
```bash
|
||||
virtualenv KnowledgeBase-Tech_env
|
||||
```
|
||||
|
||||
## Activate the Virtual Environment
|
||||
|
||||
To start using the virtual environment:
|
||||
|
||||
```bash
|
||||
source KnowledgeBase-Tech_env/bin/activate
|
||||
```
|
||||
|
||||
## Install Dependencies
|
||||
|
||||
While the environment is active, install any required packages:
|
||||
|
||||
```bash
|
||||
pip install <package-name>
|
||||
```
|
||||
|
||||
## Deactivate the Virtual Environment
|
||||
|
||||
When you're done working, deactivate the environment:
|
||||
|
||||
```bash
|
||||
deactivate
|
||||
```
|
||||
|
||||
## Working with the Environment
|
||||
|
||||
Remember to activate the `KnowledgeBase-Tech_env` environment every time you work on the KnowledgeBase-Tech Hugo site. This ensures all dependencies are isolated to this specific project.
|
||||
```
|
||||
|
||||
Replace `<package-name>` with any specific Python package you need for your Hugo site. This guide will help you maintain a clean and isolated environment for your Hugo site development.
|
||||
|
||||
---
|
||||
|
||||
**Install Python and pip (if not already installed):**
|
||||
Ubuntu 22.04 typically comes with Python 3 installed. You can check if Python is installed and its version by running:
|
||||
```bash
|
||||
python3 --version
|
||||
```
|
||||
To install pip, run:
|
||||
```bash
|
||||
sudo apt update
|
||||
sudo apt install python3-pip
|
||||
```
|
||||
|
||||
2. **Install Virtual Environment:**
|
||||
With pip installed, you can now install `virtualenv`, which is a tool for creating isolated Python environments. Run the following command:
|
||||
```bash
|
||||
pip3 install virtualenv
|
||||
```
|
||||
|
||||
3. **Create a Virtual Environment:**
|
||||
Navigate to the directory where you want to set up your Mkdocs site and create a virtual environment. Replace `myenv` with your preferred environment name.
|
||||
```bash
|
||||
virtualenv myenv
|
||||
```
|
||||
|
||||
4. **Activate the Virtual Environment:**
|
||||
To start using the virtual environment, you need to activate it. Run:
|
||||
```bash
|
||||
source myenv/bin/activate
|
||||
```
|
||||
Once activated, your command prompt should change to indicate that you are now working inside `myenv`.
|
||||
|
||||
5. **Install Mkdocs in the Virtual Environment:**
|
||||
With your virtual environment active, install Mkdocs:
|
||||
```bash
|
||||
pip install mkdocs
|
||||
```
|
||||
|
||||
6. **Create Your Mkdocs Project:**
|
||||
After Mkdocs is installed, you can create a new Mkdocs project:
|
||||
```bash
|
||||
mkdocs new my-project
|
||||
```
|
||||
Replace `my-project` with your project name. This will create a new directory with a basic configuration.
|
||||
|
||||
7. **Run Mkdocs Locally:**
|
||||
To see your Mkdocs site, navigate to your project directory and run:
|
||||
```bash
|
||||
cd my-project
|
||||
mkdocs serve
|
||||
```
|
||||
This command starts a local server. You can view your site by going to `http://127.0.0.1:8000` in your web browser.
|
||||
|
||||
8. **Deactivate the Virtual Environment:**
|
||||
When you’re done working in the virtual environment, you can deactivate it by running:
|
||||
```bash
|
||||
deactivate
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# `venv` User Guide for Python Virtual Environments
|
||||
|
||||
The `venv` module, included in Python 3, is used for creating isolated Python environments. This guide provides instructions on creating and managing virtual environments with `venv`.
|
||||
|
||||
## Creating Virtual Environments
|
||||
|
||||
### Creating the First Environment
|
||||
|
||||
1. **Navigate to your project directory:**
|
||||
```bash
|
||||
cd path/to/your/project
|
||||
```
|
||||
|
||||
2. **Create a virtual environment named `env1`:**
|
||||
```bash
|
||||
python3 -m venv env1
|
||||
```
|
||||
|
||||
3. **Activate the environment:**
|
||||
- On Windows:
|
||||
```bash
|
||||
.\env1\Scripts\activate
|
||||
```
|
||||
- On Unix or MacOS:
|
||||
```bash
|
||||
source env1/bin/activate
|
||||
```
|
||||
|
||||
### Creating the Second Environment
|
||||
|
||||
1. **Create another environment named `env2`:**
|
||||
```bash
|
||||
python3 -m venv env2
|
||||
```
|
||||
|
||||
2. **Activate the environment as shown previously.**
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Activating and Deactivating Environments
|
||||
|
||||
- **Activate** an environment before working on the project.
|
||||
- **Deactivate** when done. Just type `deactivate` in the terminal.
|
||||
|
||||
### Managing Dependencies
|
||||
|
||||
- **Install packages** using `pip` while the environment is activated.
|
||||
- **Create a `requirements.txt`** file to keep track of your dependencies.
|
||||
```bash
|
||||
pip freeze > requirements.txt
|
||||
```
|
||||
- **Install dependencies** from the file in a new environment:
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### Keeping Environments Separate
|
||||
|
||||
- **Do not commit the environment** folder to version control. Add it to `.gitignore`.
|
||||
- **Commit `requirements.txt`** to ensure consistency across different setups.
|
||||
|
||||
### Updating the Python Version
|
||||
|
||||
- If you need to update Python, create a **new environment** with the updated version.
|
||||
- Reinstall your dependencies in the new environment from `requirements.txt`.
|
||||
|
||||
## Example Workflow
|
||||
|
||||
1. **Activate the environment** when starting work.
|
||||
2. **Install and update packages** as needed.
|
||||
3. **Regularly update `requirements.txt`** to reflect new dependencies.
|
||||
4. **Deactivate the environment** when done.
|
||||
|
||||
By following these practices, you can maintain a clean and consistent development environment for your Python projects using `venv`.
|
||||
|
||||
---
|
||||
|
||||
# Guide to Python Virtual Environments
|
||||
|
||||
Python virtual environments are essential tools for managing project-specific dependencies and Python versions. This guide covers three popular tools: `virtualenv`, `venv`, and `conda`.
|
||||
|
||||
## 1. virtualenv
|
||||
|
||||
### Description
|
||||
`virtualenv` is a widely-used tool for creating isolated Python environments. It supports both Python 2 and 3 and allows different environments to have different versions of Python and packages.
|
||||
|
||||
### Pros
|
||||
- Compatible with Python 2 and 3.
|
||||
- Creates environments with different Python versions.
|
||||
- Well-documented and widely used.
|
||||
|
||||
### Cons
|
||||
- More complex than `venv`.
|
||||
- Requires installation as it's not part of the standard library.
|
||||
|
||||
### Best for
|
||||
Developers working on multiple projects with varying dependencies, especially if Python 2 support is needed.
|
||||
|
||||
## 2. venv
|
||||
|
||||
### Description
|
||||
`venv` is a module in Python 3 (3.3 and newer) for creating virtual environments, offering a streamlined approach compared to `virtualenv`.
|
||||
|
||||
### Pros
|
||||
- Part of the Python standard library (no extra installation).
|
||||
- Simpler than `virtualenv`.
|
||||
- Good for Python 3 projects.
|
||||
|
||||
### Cons
|
||||
- Only for Python 3.
|
||||
- Less flexibility in Python version management compared to `virtualenv`.
|
||||
|
||||
### Best for
|
||||
Python 3 projects where simplicity and ease of use are key, without the need for Python 2.
|
||||
|
||||
## 3. conda
|
||||
|
||||
### Description
|
||||
`conda` is a package and environment management system that supports multiple languages. It's especially popular in data science for managing complex dependencies.
|
||||
|
||||
### Pros
|
||||
- Manages both Python and non-Python packages.
|
||||
- Ideal for complex, multi-language projects.
|
||||
- Widely used in data science and machine learning.
|
||||
|
||||
### Cons
|
||||
- More complex than `venv` and `virtualenv`.
|
||||
- Overkill for simple Python-only projects.
|
||||
|
||||
### Best for
|
||||
Complex projects involving data science, machine learning, or multiple programming languages.
|
||||
|
||||
## Choosing the Right Tool
|
||||
|
||||
- **Project Complexity**: Use `venv` for simple projects, `virtualenv` for medium complexity, and `conda` for complex, multi-language projects.
|
||||
- **Ease of Use**: `venv` for straightforward Python 3 projects, `virtualenv` for more control, and `conda` for complex dependency management.
|
||||
- **Cross-Language Support**: Choose `conda` for projects requiring multi-language support.
|
||||
- **Community and Documentation**: All three have strong communities and documentation. Choose based on project needs.
|
||||
|
||||
In summary, your choice depends on the project's requirements, complexity, and language support. `venv` is suitable for most Python 3 projects, while `virtualenv` and `conda` cater to more complex scenarios.
|
||||
|
||||
---
|
||||
|
||||
# Best Practices for Structuring Python Virtual Environments
|
||||
|
||||
Organizing virtual environments is crucial for maintaining a clean and efficient workspace when working on multiple Python projects. Below are some guidelines to help structure your virtual environments effectively.
|
||||
|
||||
## 1. Project-Specific Environments
|
||||
|
||||
- **Separate Environment for Each Project**:
|
||||
Create an individual virtual environment for every project to avoid dependency conflicts.
|
||||
|
||||
- **Environment Location**:
|
||||
```plaintext
|
||||
Place the virtual environment directory inside the project's root directory.
|
||||
|
||||
Example Structure:
|
||||
MyProject/
|
||||
├── .gitignore
|
||||
├── my_project_env/
|
||||
├── src/
|
||||
├── tests/
|
||||
└── requirements.txt
|
||||
```
|
||||
|
||||
Ensure to exclude the environment directory from version control.
|
||||
|
||||
## 2. Naming Conventions
|
||||
|
||||
- **Descriptive Names**:
|
||||
Choose names that clearly identify the associated project, like `data_analyzer_env` for a "DataAnalyzer" project.
|
||||
|
||||
- **Consistency**:
|
||||
Maintain consistent naming conventions across different projects.
|
||||
|
||||
## 3. Requirements File
|
||||
|
||||
- **Use `requirements.txt`**:
|
||||
Include a `requirements.txt` file in the root directory of each project.
|
||||
|
||||
```bash
|
||||
pip freeze > requirements.txt
|
||||
```
|
||||
|
||||
## 4. Documentation
|
||||
|
||||
- **README File**:
|
||||
Add a README in your project's root, documenting the setup and activation steps for the environment.
|
||||
|
||||
## 5. Centralized Management (Optional)
|
||||
|
||||
- **Central Directory**:
|
||||
Alternatively, you can store all virtual environments in a central directory, e.g., `~/python_environments/`.
|
||||
|
||||
```plaintext
|
||||
python_environments/
|
||||
├── data_analyzer_env/
|
||||
├── web_app_env/
|
||||
└── machine_learning_env/
|
||||
```
|
||||
|
||||
- **Naming Reference**:
|
||||
Ensure the names are descriptive enough to indicate their associated project.
|
||||
|
||||
## 6. Environment Variables
|
||||
|
||||
- **.env Files**:
|
||||
Use `.env` files for environment-specific settings, loading them with libraries like `python-dotenv`.
|
||||
|
||||
## 7. Regular Maintenance
|
||||
|
||||
- **Keep Updated**:
|
||||
Regularly update the dependencies in your environments.
|
||||
|
||||
- **Cleanup**:
|
||||
Remove or archive environments for inactive projects.
|
||||
|
||||
These guidelines aim to provide a structured approach to managing Python virtual environments, enhancing clarity and efficiency in your development workflow.
|
||||
|
||||
---
|
||||
|
||||
# Managing Environment Variables in Python Virtual Environments
|
||||
|
||||
Using `.env` files for environment-specific settings is a best practice in Python development. This guide explains how to set up and use `.env` files within virtual environments.
|
||||
|
||||
## What are `.env` Files?
|
||||
|
||||
- `.env` files are simple text files that contain environment variables.
|
||||
- They are used to store configuration settings that should not be hard-coded in your code, such as API keys, database URLs, and other sensitive information.
|
||||
|
||||
## Setting Up `.env` Files
|
||||
|
||||
### 1. Creating `.env` File
|
||||
|
||||
- Place a `.env` file in your project's root directory.
|
||||
- Add environment variables in the format `KEY=value`.
|
||||
|
||||
```plaintext
|
||||
# Example .env file
|
||||
DATABASE_URL=postgresql://user:password@localhost/mydatabase
|
||||
API_KEY=yourapikey
|
||||
```
|
||||
|
||||
### 2. Using `python-dotenv` to Load Variables
|
||||
|
||||
- Install `python-dotenv` to easily load the variables from `.env` file.
|
||||
|
||||
```bash
|
||||
pip install python-dotenv
|
||||
```
|
||||
|
||||
- Import `dotenv` in your main script and load the variables.
|
||||
|
||||
```python
|
||||
from dotenv import load_dotenv
|
||||
load_dotenv()
|
||||
```
|
||||
|
||||
## Accessing Environment Variables
|
||||
|
||||
- Access variables using `os.environ`.
|
||||
|
||||
```python
|
||||
import os
|
||||
database_url = os.getenv('DATABASE_URL')
|
||||
api_key = os.getenv('API_KEY')
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
- **Never Commit `.env` Files**: Add `.env` to your `.gitignore` file to prevent sensitive information from being committed to version control.
|
||||
- **Use Different `.env` Files for Different Environments**: For example, `.env.development`, `.env.production` for different deployment stages.
|
||||
- **Keep `.env` File Updated**: Regularly update the `.env` file with any new or changed environment variables.
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- Keep your `.env` files secure and only share them with trusted team members.
|
||||
- Regularly audit the environment variables and remove any that are no longer in use.
|
||||
|
||||
By following these practices, you can securely manage environment-specific settings in your Python projects, keeping sensitive information out of your source code.
|
||||
|
||||
---
|
||||
|
||||
# Recommended Python Project Structure with Virtual Environment
|
||||
|
||||
When setting up a Python project with a virtual environment, organizing your project's directory structure effectively is crucial. Below is an expanded explanation of a typical project structure:
|
||||
|
||||
## Project Directory: MyProject
|
||||
|
||||
### `MyProject/` - The Root Directory
|
||||
- This is the main folder that contains your entire project. It's named after your project (`MyProject` in this case).
|
||||
|
||||
#### `.gitignore`
|
||||
- This file tells Git which files or folders to ignore in your project.
|
||||
- It's essential to include your virtual environment directory (`my_project_env/`) here to prevent it from being tracked by version control, as it contains system-specific settings and dependencies.
|
||||
|
||||
#### `my_project_env/`
|
||||
- This is your virtual environment directory where dependencies and settings specific to this project are stored.
|
||||
- It's created by running `python3 -m venv my_project_env` inside your root directory.
|
||||
- This environment keeps your project's dependencies isolated from the global Python installation and other projects.
|
||||
|
||||
#### `src/`
|
||||
- Short for "source", this directory contains all the source code of your project.
|
||||
- It's a good practice to separate your code into this directory to keep it organized and distinguish it from other project files.
|
||||
- Within `src/`, you can have various Python scripts and modules that make up your application.
|
||||
|
||||
#### `tests/`
|
||||
- This folder contains all the test code for your project.
|
||||
- Keeping tests in a separate directory helps in maintaining them distinctly from your actual project code.
|
||||
- It typically includes unit tests, integration tests, and other test scripts.
|
||||
|
||||
#### `requirements.txt`
|
||||
- This file lists all the Python dependencies required by your project.
|
||||
- It's created by running `pip freeze > requirements.txt` inside your activated virtual environment.
|
||||
- This file ensures that anyone who clones your project can install the exact same dependencies by running `pip install -r requirements.txt`.
|
||||
|
||||
This structure is just a guideline and can be adjusted based on the specific needs of your project. The key is to keep it organized and logical to facilitate easy navigation and collaboration.
|
||||
|
||||
---
|
||||
@@ -0,0 +1,44 @@
|
||||
# Python and JavaScript in Software Development
|
||||
|
||||
In Python and JavaScript projects, the concepts of code, runtime, and libraries are crucial, just as they are in general software development.
|
||||
|
||||
## Code
|
||||
|
||||
### Python
|
||||
- **Python Code**: Written in `.py` files using the Python programming language, an interpreted, high-level language known for its simplicity and readability.
|
||||
- **Domains**: Widely used in data analysis, web development, and machine learning.
|
||||
- **Syntax**: Utilizes indentation to define code blocks, emphasizing readability.
|
||||
|
||||
### JavaScript
|
||||
- **JavaScript Code**: Written in `.js` files, primarily used for client-side scripting in web development.
|
||||
- **Usage**: Enables interactive elements on web pages and is also used on the server-side (e.g., Node.js).
|
||||
- **Characteristics**: Event-driven and non-blocking, crucial for responsive web applications.
|
||||
|
||||
## Runtime
|
||||
|
||||
### Python Runtime
|
||||
- **CPython**: The most common Python interpreter, written in C.
|
||||
- **Alternatives**: PyPy for performance (JIT compilation) and Jython for Java integration.
|
||||
- **Functionality**: Handles tasks like memory management and object lifecycle.
|
||||
|
||||
### JavaScript Runtime
|
||||
- **V8 Engine**: Developed by Google, written in C++, and used in browsers and Node.js.
|
||||
- **Capabilities**: Manages event loops and asynchronous behavior, key to JavaScript's performance.
|
||||
|
||||
## Libraries
|
||||
|
||||
### Python Libraries
|
||||
- **Standard Library**: Included with Python, offering a wide range of functionality (math operations, data structures, file I/O).
|
||||
- **Third-Party Libraries**: NumPy (numerical computing), Pandas (data manipulation), Flask and Django (web development).
|
||||
|
||||
### JavaScript Libraries
|
||||
- **jQuery**: A third-party library for cross-browser JavaScript DOM manipulation.
|
||||
- **Modern Libraries**: React (user interfaces), Express.js (server-side applications), D3.js (data visualization).
|
||||
|
||||
## Integration and Summary
|
||||
|
||||
- **Integration**: Code written by developers is interpreted or compiled by the runtime, which interfaces with libraries for additional functionality.
|
||||
- **Applications**: Facilitates development of applications from simple scripts to complex web applications.
|
||||
- **Evolving Nature**: Regular updates to the language, runtime environments, and libraries reflect the dynamic nature of software development.
|
||||
|
||||
This document provides a comprehensive understanding of how code, runtime, and libraries work together in Python and JavaScript projects, highlighting their roles in creating powerful and versatile applications.
|
||||
150
docs/tech_docs/python/env-GnuPG.md
Normal file
150
docs/tech_docs/python/env-GnuPG.md
Normal file
@@ -0,0 +1,150 @@
|
||||
## Managing Environment Variables Securely in Python Projects
|
||||
|
||||
This guide provides a step-by-step approach to managing environment variables in Python projects on Ubuntu servers using `dotenv` for handling environment variables and GnuPG (GPG) for file encryption.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Python 3 installed on Ubuntu Server
|
||||
- Basic familiarity with terminal commands
|
||||
|
||||
### Contents
|
||||
|
||||
1. [Setting Up dotenv with .env Files](#1-setting-up-dotenv-with-env-files)
|
||||
2. [Encrypting and Decrypting .env Files with GPG](#2-encrypting-and-decrypting-env-files-with-gpg)
|
||||
3. [Automating Decryption in Python Scripts](#3-automating-decryption-in-python-scripts)
|
||||
4. [Backing Up GPG Keys](#4-backing-up-gpg-keys)
|
||||
5. [Basic GPG Commands](#5-basic-gpg-commands)
|
||||
|
||||
---
|
||||
|
||||
### 1. Setting Up `dotenv` with `.env` Files
|
||||
|
||||
`dotenv` is a module that loads environment variables from a `.env` file into `os.environ`. This section covers creating a `.env` file and integrating `dotenv` into your Python project.
|
||||
|
||||
#### Steps:
|
||||
|
||||
1. **Create a `.env` File**:
|
||||
|
||||
```bash
|
||||
# Navigate to your project directory
|
||||
cd /path/to/your/project
|
||||
|
||||
# Create a .env file
|
||||
touch .env
|
||||
|
||||
# Add environment variables
|
||||
echo "API_KEY=yourapikey123" >> .env
|
||||
echo "DB_PASSWORD=hunter2" >> .env
|
||||
```
|
||||
|
||||
2. **Install `python-dotenv`**:
|
||||
|
||||
```bash
|
||||
pip3 install python-dotenv
|
||||
```
|
||||
|
||||
3. **Modify Your Python Script**:
|
||||
|
||||
```python
|
||||
import os
|
||||
from dotenv import load_dotenv
|
||||
|
||||
load_dotenv() # Loads the .env file
|
||||
|
||||
api_key = os.getenv('API_KEY')
|
||||
db_password = os.getenv('DB_PASSWORD')
|
||||
|
||||
# Your script continues here...
|
||||
```
|
||||
|
||||
4. **Update `.gitignore`**:
|
||||
|
||||
```bash
|
||||
echo ".env" >> .gitignore
|
||||
```
|
||||
|
||||
### 2. Encrypting and Decrypting .env Files with GPG
|
||||
|
||||
GnuPG (GPG) is used for encrypting files, ensuring sensitive information like environment variables in `.env` files is secure.
|
||||
|
||||
#### Steps:
|
||||
|
||||
1. **Install GnuPG**:
|
||||
|
||||
```bash
|
||||
sudo apt-get update
|
||||
sudo apt-get install gnupg
|
||||
```
|
||||
|
||||
2. **Encrypt the `.env` File**:
|
||||
|
||||
```bash
|
||||
gpg --encrypt --recipient your_user_id .env
|
||||
```
|
||||
|
||||
3. **Decrypt the `.env` File When Needed**:
|
||||
|
||||
```bash
|
||||
gpg --output .env --decrypt .env.gpg
|
||||
```
|
||||
|
||||
### 3. Automating Decryption in Python Scripts
|
||||
|
||||
Automate the decryption of the `.env` file at the start of your Python script for convenience while maintaining security.
|
||||
|
||||
#### Example Function:
|
||||
|
||||
```python
|
||||
import subprocess
|
||||
import os
|
||||
from dotenv import load_dotenv
|
||||
|
||||
def decrypt_env_file():
|
||||
subprocess.run(['gpg', '--quiet', '--batch', '--yes', '--decrypt', '--output', '.env', '.env.gpg'])
|
||||
|
||||
# Decrypt the .env file
|
||||
decrypt_env_file()
|
||||
|
||||
# Load environment variables
|
||||
load_dotenv()
|
||||
|
||||
# Your main script logic
|
||||
|
||||
# Delete the .env file securely after use
|
||||
os.remove('.env')
|
||||
```
|
||||
|
||||
### 4. Backing Up GPG Keys
|
||||
|
||||
Regular backups of GPG keys are essential to avoid losing access to encrypted data.
|
||||
|
||||
#### Steps:
|
||||
|
||||
1. **Export Your Private Key**:
|
||||
|
||||
```bash
|
||||
gpg --export-secret-keys your_user_id > myprivatekey.asc
|
||||
```
|
||||
|
||||
2. **Export Your Public Key**:
|
||||
|
||||
```bash
|
||||
gpg --export your_user_id > mypublickey.asc
|
||||
```
|
||||
|
||||
### 5. Basic GPG Commands
|
||||
|
||||
Familiarize yourself with basic GPG commands for managing your keys and encrypted files.
|
||||
|
||||
#### Common Commands:
|
||||
|
||||
- **List Keys**: `gpg --list-keys`, `gpg --list-secret-keys`
|
||||
- **Import Key**: `gpg --import [file]`
|
||||
- **Export Key**: `gpg --export -a [email/id] > public.key`
|
||||
- **Delete Key**: `gpg --delete-key [email/id]`, `gpg --delete-secret-key [email/id]`
|
||||
- **Encrypt File**: `gpg --encrypt --recipient [email/id] [file]`
|
||||
- **Decrypt File**: `gpg --decrypt [file.gpg]`
|
||||
|
||||
---
|
||||
|
||||
By following this guide, you will be able to securely manage environment variables in your Python projects, leveraging `dotenv` for environment variable management and GnuPG for encryption, ensuring your sensitive data remains protected.
|
||||
Reference in New Issue
Block a user