mirror of
https://github.com/Rainyy21/framework_note.git
synced 2026-10-10 23:30:28 -04:00
vault backup: 2026-05-30 16:02:33
This commit is contained in:
Rainyy21
@@ -0,0 +1,10 @@
|
||||
---
|
||||
tags:
|
||||
- coding
|
||||
---
|
||||
```dataview
|
||||
TABLE file.tags AS Tags, file.mtime AS Modified
|
||||
FROM "devop_note"
|
||||
WHERE file.name != "00_devop_note"
|
||||
SORT file.name ASC
|
||||
```
|
||||
@@ -0,0 +1,6 @@
|
||||
---
|
||||
aliases:
|
||||
up: "[[00_devop_note]]"
|
||||
tags:
|
||||
- devop
|
||||
---
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
tags:
|
||||
- devop
|
||||
up: "[[00_devop_note]]"
|
||||
---
|
||||
@@ -0,0 +1,8 @@
|
||||
---
|
||||
tags:
|
||||
- devop
|
||||
up: "[[00_devop_note]]"
|
||||
---
|
||||
# What is Maven?
|
||||
maven is a build automation tool used primarily for java projects, hosted by Apache Software Foundation. Maven projects are configured using Project Object Model (POM) in a `pom.xml` file. It's used by over 70% of Java organizations, so employers actively seek people with strong Maven skills
|
||||
|
||||
@@ -0,0 +1,175 @@
|
||||
---
|
||||
aliases:
|
||||
up: "[[00_devop_note]]"
|
||||
tags:
|
||||
- devop
|
||||
---
|
||||
|
||||
# Uvicorn and Gunicorn
|
||||
|
||||
Both are Python application servers — they run your Python web app and hand requests back and forth with a web server like Nginx. But they solve **different problems**, and you'll often see them used **together**.
|
||||
|
||||
The key split: **Gunicorn is WSGI** (synchronous), **Uvicorn is ASGI** (asynchronous).
|
||||
|
||||
## Quick refresher: WSGI vs ASGI
|
||||
|
||||
| | WSGI | ASGI |
|
||||
|---|---|---|
|
||||
| Spec | PEP 3333 | "Asynchronous Server Gateway Interface" |
|
||||
| Model | Sync, one request per worker at a time | Async, many concurrent requests per worker |
|
||||
| Frameworks | Django (classic), Flask | FastAPI, Starlette, Django (async views), Sanic |
|
||||
| Supports WebSockets? | No | Yes |
|
||||
| Supports HTTP/2, SSE? | No | Yes |
|
||||
|
||||
If your app uses `async def` view functions or WebSockets, you need ASGI.
|
||||
If it's a traditional Django/Flask app with `def` views, WSGI is fine.
|
||||
|
||||
---
|
||||
|
||||
## Gunicorn ("Green Unicorn")
|
||||
|
||||
A **WSGI** server. Mature, simple, battle-tested. The de facto default for Django/Flask in production.
|
||||
|
||||
### Why people pick it
|
||||
|
||||
- **Simple config** — most options are sensible by default
|
||||
- **Pre-fork worker model** — master process forks N workers; each worker handles one request at a time
|
||||
- **Stable** — has been the standard for years
|
||||
- **Good signal handling** — graceful reloads, zero-downtime restarts
|
||||
|
||||
### Minimal usage
|
||||
|
||||
```bash
|
||||
gunicorn myproject.wsgi:application --workers 4 --bind 0.0.0.0:8000
|
||||
```
|
||||
|
||||
Or with a config file `gunicorn.conf.py`:
|
||||
|
||||
```python
|
||||
bind = "unix:/tmp/myproject.sock"
|
||||
workers = 4
|
||||
worker_class = "sync" # default
|
||||
timeout = 30
|
||||
accesslog = "-"
|
||||
errorlog = "-"
|
||||
```
|
||||
|
||||
### Worker classes
|
||||
|
||||
Gunicorn lets you swap the worker type:
|
||||
|
||||
- `sync` — default, one request at a time per worker
|
||||
- `gthread` — threaded workers (good for I/O-bound apps)
|
||||
- `gevent` / `eventlet` — async via greenlets (legacy)
|
||||
- `uvicorn.workers.UvicornWorker` — **this is the bridge to ASGI** (see below)
|
||||
|
||||
### How many workers?
|
||||
|
||||
Rule of thumb: `(2 × CPU cores) + 1`. So a 4-core box → 9 workers.
|
||||
|
||||
---
|
||||
|
||||
## Uvicorn
|
||||
|
||||
An **ASGI** server built on `uvloop` and `httptools` — both written in C, which makes it very fast. It's the standard server for FastAPI and modern async Python web apps.
|
||||
|
||||
### Why people pick it
|
||||
|
||||
- **Async-native** — handles thousands of concurrent connections per worker
|
||||
- **WebSockets + HTTP/2 support**
|
||||
- **Very fast** — uvloop is a drop-in faster replacement for asyncio's event loop
|
||||
- **Lightweight** — small dependency surface
|
||||
|
||||
### Minimal usage
|
||||
|
||||
```bash
|
||||
uvicorn myproject.main:app --host 0.0.0.0 --port 8000
|
||||
```
|
||||
|
||||
For development with auto-reload:
|
||||
|
||||
```bash
|
||||
uvicorn myproject.main:app --reload
|
||||
```
|
||||
|
||||
### Where it falls short alone
|
||||
|
||||
Uvicorn by itself is **single-process**. To use multiple CPU cores in production, you need to either:
|
||||
|
||||
1. Run multiple Uvicorn instances behind a load balancer, OR
|
||||
2. Run Uvicorn **inside Gunicorn** as worker processes (the common pattern)
|
||||
|
||||
---
|
||||
|
||||
## The common production combo: Gunicorn + Uvicorn
|
||||
|
||||
This is the standard FastAPI production setup:
|
||||
|
||||
```bash
|
||||
gunicorn myproject.main:app \
|
||||
--workers 4 \
|
||||
--worker-class uvicorn.workers.UvicornWorker \
|
||||
--bind 0.0.0.0:8000
|
||||
```
|
||||
|
||||
What's happening:
|
||||
|
||||
- **Gunicorn** is the process manager — forks workers, handles signals, restarts dead workers, manages graceful shutdowns
|
||||
- **Uvicorn** runs *inside* each Gunicorn worker — provides the ASGI event loop that runs your async code
|
||||
|
||||
You get the best of both: Gunicorn's robust process management + Uvicorn's async performance.
|
||||
|
||||
### Visual
|
||||
|
||||
```
|
||||
┌─ Uvicorn worker (async loop) ─ FastAPI app
|
||||
│
|
||||
Nginx → Gunicorn ├─ Uvicorn worker (async loop) ─ FastAPI app
|
||||
(master) │
|
||||
├─ Uvicorn worker (async loop) ─ FastAPI app
|
||||
│
|
||||
└─ Uvicorn worker (async loop) ─ FastAPI app
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Decision guide
|
||||
|
||||
| Your situation | Use |
|
||||
|---|---|
|
||||
| Django (sync), Flask | **Gunicorn** alone |
|
||||
| FastAPI, Starlette, async Django | **Gunicorn + UvicornWorker** |
|
||||
| Local dev, single FastAPI process | **Uvicorn** alone (`--reload`) |
|
||||
| WebSockets required | Must be ASGI → **Uvicorn** (alone or under Gunicorn) |
|
||||
| Legacy app, uWSGI already configured | [[uWSGI]] — no urgent need to migrate |
|
||||
|
||||
---
|
||||
|
||||
## Comparison with [[uWSGI]]
|
||||
|
||||
| | Gunicorn | Uvicorn | uWSGI |
|
||||
|---|---|---|---|
|
||||
| Protocol | WSGI | ASGI | WSGI (+ many others) |
|
||||
| Async support | No (sync workers) | Yes (native) | Limited |
|
||||
| Config complexity | Low | Low | **Very high** |
|
||||
| WebSockets | No | Yes | Partial |
|
||||
| Speed (raw) | Good | **Fastest** for async | Fast but heavy |
|
||||
| Maintained actively | Yes | Yes | Concerns |
|
||||
| Best for | Django/Flask | FastAPI | Legacy / specialized features |
|
||||
|
||||
---
|
||||
|
||||
## Common gotchas
|
||||
|
||||
1. **Don't run Uvicorn `--reload` in production** — it's a dev-only feature, has overhead and isn't safe.
|
||||
2. **Workers ≠ threads** — each Gunicorn worker is a separate Python process with its own memory. Database connections, in-memory caches, etc. are *not* shared between workers.
|
||||
3. **Timeouts matter** — Gunicorn's default `timeout=30s` will kill workers running long async tasks. Tune it for your workload.
|
||||
4. **Nginx is still recommended in front** — Uvicorn/Gunicorn don't do SSL termination, static files, or rate limiting as well as Nginx.
|
||||
5. **Logging** — by default both log to stdout/stderr; route to your log aggregator via container stdout (`-` in config).
|
||||
|
||||
---
|
||||
|
||||
## Related
|
||||
|
||||
- [[uWSGI]] — older alternative, mostly WSGI
|
||||
- [[Apache Tomcat]] — the Java equivalent (servlet container)
|
||||
@@ -0,0 +1,94 @@
|
||||
---
|
||||
aliases:
|
||||
up: "[[00_devop_note]]"
|
||||
tags:
|
||||
- devop
|
||||
---
|
||||
w
|
||||
# What is uWSGI?
|
||||
|
||||
**uWSGI** is an application server that sits between a web server (like Nginx or Apache) and a Python web application (like Django, Flask, or FastAPI). It runs your Python code and handles incoming web requests.
|
||||
|
||||
The name comes from **WSGI** (Web Server Gateway Interface), which is the standard Python spec (PEP 3333) that defines how web servers talk to Python applications. The lowercase "u" is the Greek letter μ (micro), suggesting it's lightweight — though in practice it grew into a full-featured server.
|
||||
|
||||
## Why do we need it?
|
||||
|
||||
A plain web server like Nginx doesn't know how to execute Python code. It only serves static files (HTML, CSS, images) and forwards dynamic requests elsewhere. You need a process that:
|
||||
|
||||
1. Loads your Python application into memory
|
||||
2. Receives requests from the web server
|
||||
3. Calls your Python code with the request
|
||||
4. Returns the response back to the web server
|
||||
|
||||
That middle process is uWSGI (or alternatives like Gunicorn).
|
||||
|
||||
## The typical stack
|
||||
|
||||
```
|
||||
Browser → Nginx → uWSGI → Python app (Django/Flask)
|
||||
```
|
||||
|
||||
- **Nginx**: handles SSL, static files, load balancing, gzip, caching
|
||||
- **uWSGI**: runs Python workers, manages processes/threads
|
||||
- **Python app**: your business logic
|
||||
|
||||
Nginx and uWSGI usually talk over a **Unix socket** (fast, local) or a TCP port. The protocol between them is called the **uwsgi protocol** (lowercase) — a binary protocol that's faster than plain HTTP.
|
||||
|
||||
## Key features
|
||||
|
||||
- **Process management**: spawns multiple worker processes to handle concurrent requests
|
||||
- **Threading**: each worker can run multiple threads
|
||||
- **Auto-reload**: restart workers when code changes (dev mode)
|
||||
- **Emperor mode**: one master process supervising many vassal apps
|
||||
- **Cheaper mode**: dynamically scale workers up/down based on load
|
||||
- **Multi-language**: despite the name, also supports Ruby, Perl, Go, etc.
|
||||
|
||||
## Minimal config example
|
||||
|
||||
A typical `uwsgi.ini`:
|
||||
|
||||
```ini
|
||||
[uwsgi]
|
||||
module = myproject.wsgi:application
|
||||
master = true
|
||||
processes = 4
|
||||
threads = 2
|
||||
socket = /tmp/myproject.sock
|
||||
chmod-socket = 660
|
||||
vacuum = true
|
||||
die-on-term = true
|
||||
```
|
||||
|
||||
- `module`: entry point (the WSGI callable)
|
||||
- `processes`: how many worker processes to fork
|
||||
- `socket`: where Nginx connects to
|
||||
- `vacuum`: clean up the socket on exit
|
||||
- `die-on-term`: shut down cleanly on SIGTERM
|
||||
|
||||
## Running it
|
||||
|
||||
```bash
|
||||
uwsgi --ini uwsgi.ini
|
||||
```
|
||||
|
||||
Or in production, run it under **systemd** so it restarts on failure.
|
||||
|
||||
## uWSGI vs Gunicorn
|
||||
|
||||
Both are WSGI servers. The community has largely shifted toward **Gunicorn** because:
|
||||
|
||||
- Simpler config
|
||||
- Fewer footguns
|
||||
- Easier to deploy
|
||||
|
||||
uWSGI is more feature-rich and faster in some benchmarks, but its config surface is huge (hundreds of options) and the project has had governance/maintenance concerns. For new projects, Gunicorn behind Nginx is the common default.
|
||||
|
||||
## When you'll see uWSGI
|
||||
|
||||
- Legacy Django/Flask deployments
|
||||
- Setups that need uWSGI-specific features (Emperor, cheaper, etc.)
|
||||
- Docker images for older Python web apps
|
||||
|
||||
## Related
|
||||
|
||||
- [[Apache Tomcat]] — the Java equivalent role (servlet container running Java apps behind a web server)
|
||||
@@ -0,0 +1,55 @@
|
||||
---
|
||||
tags:
|
||||
- devop
|
||||
- video
|
||||
- poetry
|
||||
- python
|
||||
up: "[[00_devop_note]]"
|
||||
related:
|
||||
- "[[Poetry]]"
|
||||
- "[[poetry_guide]]"
|
||||
- "[[poetry_project_ideas]]"
|
||||
source:
|
||||
author:
|
||||
published:
|
||||
---
|
||||
# Poetry: Dependency Management for Python
|
||||
|
||||
Poetry is a tool for **dependency management** and **packaging** in Python. It allows you to declare the libraries your project depends on and it will manage (install/update) them for you. Poetry offers a lockfile to ensure repeatable installs, and can build your project for distribution.
|
||||
|
||||
## Similarities to Maven (Java)
|
||||
|
||||
If you are familiar with Apache Maven, you can think of Poetry as serving a similar purpose in the Python ecosystem:
|
||||
|
||||
| Feature | Maven (Java) | Poetry (Python) |
|
||||
| :--- | :--- | :--- |
|
||||
| **Project Configuration** | `pom.xml` | `pyproject.toml` |
|
||||
| **Dependency Resolution** | Resolves transitive dependencies | Resolves transitive dependencies with a deterministic solver |
|
||||
| **Locking** | No direct equivalent (uses version ranges in POM) | `poetry.lock` (ensures exact versions) |
|
||||
| **Build & Packaging** | Builds JAR/WAR files | Builds Wheel and sdist packages |
|
||||
| **Publishing** | Deploys to Central/Nexus | Publishes to PyPI or private repositories |
|
||||
| **Environment Management**| Relies on external JRE/JDK | Manages Virtual Environments automatically |
|
||||
|
||||
## Key Concepts
|
||||
|
||||
### 1. `pyproject.toml`
|
||||
This is the single source of truth for your project. It replaces `setup.py`, `requirements.txt`, `setup.cfg`, `MANIFEST.in` and `pipfile`.
|
||||
|
||||
### 2. Deterministic Resolution
|
||||
Poetry comes with a custom dependency resolver that will always find a solution if one exists, or clearly explain why it failed.
|
||||
|
||||
### 3. Isolation by Default
|
||||
Poetry always runs in isolation. It either uses your existing virtual environment or creates its own to ensure that your project dependencies don't leak into your global Python installation.
|
||||
|
||||
## Basic Commands
|
||||
|
||||
- `poetry init`: Interactively create a `pyproject.toml` file.
|
||||
- `poetry add <package>`: Adds a dependency to `pyproject.toml` and installs it.
|
||||
- `poetry install`: Installs the dependencies specified in `pyproject.toml` (or `poetry.lock` if present).
|
||||
- `poetry update`: Updates dependencies to their latest versions according to `pyproject.toml` and updates the lock file.
|
||||
- `poetry run <command>`: Runs a command within the project's virtual environment.
|
||||
- `poetry shell`: Spawns a shell within the virtual environment.
|
||||
|
||||
## Why use Poetry over Pip?
|
||||
|
||||
While `pip` is the standard package installer, it doesn't handle dependency resolution or project metadata as comprehensively as Poetry. Poetry provides a more "all-in-one" experience, much like Maven does for Java, by combining dependency management, environment isolation, and packaging into a single tool.
|
||||
@@ -0,0 +1,87 @@
|
||||
---
|
||||
tags:
|
||||
- devop
|
||||
- video
|
||||
- maven
|
||||
- java
|
||||
up: "[[00_devop_note]]"
|
||||
related: "[[Maven]]"
|
||||
source: https://www.youtube.com/watch?v=T00NKLQvwYE
|
||||
author: Cameron McKenzie (TheServerSide)
|
||||
published: 2023-08-27
|
||||
---
|
||||
# Learn Apache Maven Full Tutorial in Java for Beginners
|
||||
|
||||
> An hour-long beginner-friendly course that walks through Apache Maven from scratch — installation, configuration, core commands, dependencies, plugins, and integrations with Jenkins and Docker.
|
||||
|
||||
## Overview
|
||||
Apache Maven is the most widely used build automation tool in the Java ecosystem (used by ~70% of Java organizations). This tutorial progresses from foundational concepts (installing Java/JDK and Maven) through to advanced topics like building cloud-native microservices and CI/CD integration.
|
||||
|
||||
## Topics Covered
|
||||
|
||||
### 1. Getting Started
|
||||
- What Maven is and what it enables developers to do
|
||||
- Installing the JDK (prerequisite)
|
||||
- Downloading, installing, and configuring Maven
|
||||
- Verifying the install with `mvn -v`
|
||||
|
||||
### 2. Core Maven Commands
|
||||
- `mvn compile` — compile source code
|
||||
- `mvn test` — run unit tests
|
||||
- `mvn package` — bundle compiled code into a JAR/WAR
|
||||
- `mvn install` — install artifact into local repo
|
||||
- `mvn deploy` — push artifact to a remote repo
|
||||
- `mvn clean` — wipe the `target/` directory
|
||||
|
||||
### 3. The POM File (`pom.xml`)
|
||||
- The heart of every Maven project
|
||||
- POM types (parent, aggregator, effective POM)
|
||||
- Properties and how they control build behavior
|
||||
- Project coordinates: `groupId`, `artifactId`, `version`
|
||||
|
||||
### 4. Dependency Management
|
||||
- Declaring dependencies
|
||||
- **Scopes**: `compile`, `provided`, `runtime`, `test`, `system`
|
||||
- External dependencies
|
||||
- Exclusions and optional dependencies
|
||||
- How Maven resolves transitive dependencies
|
||||
|
||||
### 5. Plugins
|
||||
- Plugins extend Maven's core functionality
|
||||
- Commonly used:
|
||||
- **Compiler Plugin** — controls the Java source/target version
|
||||
- **Surefire Plugin** — runs unit tests
|
||||
- When to add a plugin vs. rely on defaults
|
||||
|
||||
### 6. Maven vs. Gradle
|
||||
- Comparison of the two dominant JVM build tools
|
||||
- Tradeoffs: XML configuration (Maven) vs. Groovy/Kotlin DSL (Gradle)
|
||||
- Why Maven still dominates enterprise Java
|
||||
|
||||
### 7. IDE Integration
|
||||
- Creating and managing Maven projects in **IntelliJ IDEA** and **Eclipse**
|
||||
- Building projects from the IDE
|
||||
- Creating executable JARs
|
||||
- Handling multi-module projects
|
||||
|
||||
### 8. DevOps Integration
|
||||
- **Maven + Jenkins** — running builds in CI pipelines
|
||||
- **Maven + Docker** — packaging artifacts into Docker images
|
||||
- Building cloud-native microservices
|
||||
|
||||
## Key Takeaways
|
||||
- Maven is *opinionated* — follow the standard directory layout (`src/main/java`, `src/test/java`) and most things just work
|
||||
- The `pom.xml` is declarative; you describe *what* the project is, not *how* to build it
|
||||
- Dependency scopes matter — using `test` scope keeps test libraries out of production artifacts
|
||||
- Plugins are how Maven gets extended; nearly every advanced behavior is plugin-driven
|
||||
|
||||
## Actionable Next Steps
|
||||
- [ ] Install JDK and Maven; verify with `mvn -v`
|
||||
- [ ] Generate a starter project: `mvn archetype:generate`
|
||||
- [ ] Try the full lifecycle: `mvn clean package`
|
||||
- [ ] Add a dependency from [Maven Central](https://search.maven.org)
|
||||
- [ ] Wire a simple project into Jenkins or Docker
|
||||
|
||||
## Source
|
||||
- Video: [Learn Apache Maven Full Tutorial in Java for Beginners](https://www.youtube.com/watch?v=T00NKLQvwYE)
|
||||
- Companion article: [TheServerSide — Learn Maven tutorial for beginners](https://www.theserverside.com/video/Learn-Maven-tutorial-for-beginners)
|
||||
Reference in New Issue
Block a user