grokaem-konkurentnost.pdf
8.9 MB
Хорошая серия книг с помощью которой я подучил алгоритмы. Теперь из той же серии но про конкурентность в программировании. Автор Кирилл Бобров
Под капотом индексов в Postgres
https://habr.com/ru/companies/vk/articles/261871/
https://habr.com/ru/companies/vk/articles/261871/
Хабр
«Под капотом» индексов Postgres
Капитан Немо у штурвала «Наутилуса» Индексы — один из самых мощных инструментов в реляционных базах данных. Мы используем их, когда нужно быстро найти какие-то значения, когда объединяем базы данных,...
Иногда нужно сгенерировать случайные данные (random data). У меня частенько бывает что нужно создать каких либо данных чтобы записать в БД например. Эта та самая "либра" которая позволяет легко это делать!
На борту набор:
Генерация русских значений: ИНН, ОКПО, КПП и т.д.
БИК (Faker::Russian.bik)
ИНН (Faker::Russian.inn)
ОКПО (Faker::Russian.okpo)
КПП (Faker::Russian.kpp)
ОГРН (Faker::Russian.ogrn)
Корреспондентский счёт (Faker::Russian.correspondent_account)
Расчётный счёт (Faker::Russian.rs)
OKATO (Faker::Russian.okato)
СНИЛС (Faker::Russian.snils)
(TODO) Значение паспорта (Faker::Russian.passport)
Кадастровый номер (Faker::Russian.cadastral_number)
Для проверки сгенерированных значений можешь использовать gem 'validates_russian'
Faker generate random data: RU
оригинальный Faker тут:
https://github.com/joke2k/faker
https://github.com/asiniy/faker-russian
На борту набор:
Генерация русских значений: ИНН, ОКПО, КПП и т.д.
БИК (Faker::Russian.bik)
ИНН (Faker::Russian.inn)
ОКПО (Faker::Russian.okpo)
КПП (Faker::Russian.kpp)
ОГРН (Faker::Russian.ogrn)
Корреспондентский счёт (Faker::Russian.correspondent_account)
Расчётный счёт (Faker::Russian.rs)
OKATO (Faker::Russian.okato)
СНИЛС (Faker::Russian.snils)
(TODO) Значение паспорта (Faker::Russian.passport)
Кадастровый номер (Faker::Russian.cadastral_number)
Для проверки сгенерированных значений можешь использовать gem 'validates_russian'
Faker generate random data: RU
оригинальный Faker тут:
https://github.com/joke2k/faker
https://github.com/asiniy/faker-russian
GitHub
GitHub - asiniy/faker-russian: Faker russian specific values. INN, OKPO, OGRN et.c.
Faker russian specific values. INN, OKPO, OGRN et.c. - asiniy/faker-russian
https://docs.allauth.org/en/dev/introduction/index.html
Django-allauth
Features
🔑 Comprehensive account functionality
Supports multiple authentication schemes (e.g. login by user name, or by email), as well as multiple strategies for account verification (ranging from none to mandatory email verification).
👥 Social Login
Login using external identity providers, supporting any Open ID Connect compatible provider, many OAuth 1.0/2.0 providers, as well as custom protocols such as, for example, Telegram authentication.
💼 Enterprise ready
Supports SAML 2.0, which is often used in a B2B context.
🕵️ Battle-tested
The package has been out in the open since 2010. It is in use by many commercial companies whose business depends on it and has hence been subjected to various penetration testing attempts.
⏳Rate limiting
When you expose an authentication-enabled web service to the internet, it is important to be prepared for potential brute force attempts. Therefore, rate limiting is enabled out of the box.
🔒 Private
Many sites leak information. For example, on many sites you can check whether someone you know has an account by input their email address into the password forgotten form, or trying to signup with it. We offer account enumeration prevention, making it impossible to tell whether or not somebody already has an account.
🧩 Customizable
As a developer, you have the flexibility to customize the core functionality according to your specific requirements. By employing the adapter pattern, you can effortlessly introduce interventions at the desired points to deviate from the standard behavior. This level of customization empowers you to tailor the software to meet your unique needs and preferences.
Django-allauth
Features
🔑 Comprehensive account functionality
Supports multiple authentication schemes (e.g. login by user name, or by email), as well as multiple strategies for account verification (ranging from none to mandatory email verification).
👥 Social Login
Login using external identity providers, supporting any Open ID Connect compatible provider, many OAuth 1.0/2.0 providers, as well as custom protocols such as, for example, Telegram authentication.
💼 Enterprise ready
Supports SAML 2.0, which is often used in a B2B context.
🕵️ Battle-tested
The package has been out in the open since 2010. It is in use by many commercial companies whose business depends on it and has hence been subjected to various penetration testing attempts.
⏳Rate limiting
When you expose an authentication-enabled web service to the internet, it is important to be prepared for potential brute force attempts. Therefore, rate limiting is enabled out of the box.
🔒 Private
Many sites leak information. For example, on many sites you can check whether someone you know has an account by input their email address into the password forgotten form, or trying to signup with it. We offer account enumeration prevention, making it impossible to tell whether or not somebody already has an account.
🧩 Customizable
As a developer, you have the flexibility to customize the core functionality according to your specific requirements. By employing the adapter pattern, you can effortlessly introduce interventions at the desired points to deviate from the standard behavior. This level of customization empowers you to tailor the software to meet your unique needs and preferences.
Partitioning:
Definition: Partitioning involves dividing a single table (or index) within a single database instance into smaller, more manageable pieces called partitions. These partitions are still part of the same logical table but are stored separately.
Purpose: Primarily used to improve performance, manageability, and maintenance within a single database system. It can speed up queries by allowing the database to scan only relevant partitions and facilitate tasks like backups and data archiving.
Scope: Operates within the confines of a single database server or instance.
Sharding:
Definition: Sharding, a form of horizontal partitioning, involves distributing data across multiple independent database servers or instances, each holding a subset of the entire dataset. Each of these independent instances is called a "shard."
Purpose: Primarily used for horizontal scalability, enabling a system to handle larger datasets and higher transaction volumes than a single server can manage. It distributes the workload across multiple machines, preventing any single server from becoming a bottleneck.
Scope: Operates across a distributed system of multiple database servers.
Key Differences Summarized:
Feature
Partitioning
Sharding
Scope
Within a single database instance
Across multiple database servers/instances
Goal
Performance optimization, manageability within a single server
Horizontal scalability, distributed workload
Data Distribution
Logical division of a table into smaller parts
Physical distribution of data across different servers
Complexity
Generally less complex to implement and manage
Significantly more
Definition: Partitioning involves dividing a single table (or index) within a single database instance into smaller, more manageable pieces called partitions. These partitions are still part of the same logical table but are stored separately.
Purpose: Primarily used to improve performance, manageability, and maintenance within a single database system. It can speed up queries by allowing the database to scan only relevant partitions and facilitate tasks like backups and data archiving.
Scope: Operates within the confines of a single database server or instance.
Sharding:
Definition: Sharding, a form of horizontal partitioning, involves distributing data across multiple independent database servers or instances, each holding a subset of the entire dataset. Each of these independent instances is called a "shard."
Purpose: Primarily used for horizontal scalability, enabling a system to handle larger datasets and higher transaction volumes than a single server can manage. It distributes the workload across multiple machines, preventing any single server from becoming a bottleneck.
Scope: Operates across a distributed system of multiple database servers.
Key Differences Summarized:
Feature
Partitioning
Sharding
Scope
Within a single database instance
Across multiple database servers/instances
Goal
Performance optimization, manageability within a single server
Horizontal scalability, distributed workload
Data Distribution
Logical division of a table into smaller parts
Physical distribution of data across different servers
Complexity
Generally less complex to implement and manage
Significantly more
Немного ясности про сериализацию и де сериализацию, парсинг...
Лично у меня часто путались слова "парсинг", " сериализация".
Сериализация это больше про объект в байты, и де сериализация, обратный процесс, парсинг больше про HTML или JSON. Последние надо обходить и анализировать, извлекать нужное.
---
Парсинг — это процесс извлечения и структурирования информации из текста или данных, тогда как сериализация — это преобразование объекта в формат (часто строку) для сохранения или передачи. Парсинг анализирует данные, а сериализация — преобразует их в поток, который затем может быть десериализован обратно в объект.
Лично у меня часто путались слова "парсинг", " сериализация".
Сериализация это больше про объект в байты, и де сериализация, обратный процесс, парсинг больше про HTML или JSON. Последние надо обходить и анализировать, извлекать нужное.
---
Парсинг — это процесс извлечения и структурирования информации из текста или данных, тогда как сериализация — это преобразование объекта в формат (часто строку) для сохранения или передачи. Парсинг анализирует данные, а сериализация — преобразует их в поток, который затем может быть десериализован обратно в объект.
Forwarded from DevOps
Автоматизируй всё с Ansible!
Ansible — это мощный инструмент, который упрощает управление конфигурацией, развертывание приложений и оркестрацию задач. Статья рассказывает о лучших практиках использования Ansible и о том, как автоматизировать повседневные задачи, экономя время и силы.
https://agralrst.medium.com/automate-everything-with-ansible-aac7eb4d5cf9
📲 Мы в MAX
#devops #девопс
Подпишись 👉@i_DevOps
Ansible — это мощный инструмент, который упрощает управление конфигурацией, развертывание приложений и оркестрацию задач. Статья рассказывает о лучших практиках использования Ansible и о том, как автоматизировать повседневные задачи, экономя время и силы.
https://agralrst.medium.com/automate-everything-with-ansible-aac7eb4d5cf9
#devops #девопс
Подпишись 👉@i_DevOps
Please open Telegram to view this post
VIEW IN TELEGRAM
В настоящий момент я ищу вакансию программиста-разработчика.
Имею опыт разработки Rest API на Python (Django, FastAPI, DRF), модели (LangChain), контейнеризации (Docker) и работы с сетевой инфраструктурой.
Буду рад, если вы ознакомитесь с моим резюме и портфолио — уверен, они покажутся вам интересными.
—-
Резюме и портфолио:
Github:
https://github.com/ivan-telepop
Blog & Web Site:
https://ivan-telepop.github.io/#russ
Telegram:
@ewanG808
Имею опыт разработки Rest API на Python (Django, FastAPI, DRF), модели (LangChain), контейнеризации (Docker) и работы с сетевой инфраструктурой.
Буду рад, если вы ознакомитесь с моим резюме и портфолио — уверен, они покажутся вам интересными.
—-
Резюме и портфолио:
Github:
https://github.com/ivan-telepop
Blog & Web Site:
https://ivan-telepop.github.io/#russ
Telegram:
@ewanG808
Need to read about deps with pip
https://realpython.com/what-is-pip/#installing-packages-in-editable-mode-to-ease-development
https://realpython.com/what-is-pip/#installing-packages-in-editable-mode-to-ease-development
Realpython
Using Python's pip to Manage Your Projects' Dependencies – Real Python
What is pip? In this beginner-friendly tutorial, you'll learn how to use pip, the standard package manager for Python, so that you can install and manage packages that aren't part of the Python standard library.
-o, --output <path>Lock file name (default=pylock.toml). Use - for stdout.
(environment variable:
PIP_OUTPUT)-r, --requirement <file>
Install from the given requirements file. This option can be used multiple times.
(environment variable:
PIP_REQUIREMENT)-c, --constraint <file>
Constrain versions using the given constraints file. This option can be used multiple times.
(environment variable:
PIP_CONSTRAINT)--build-constraint <file>
Constrain build dependencies using the given constraints file. This option can be used multiple times.
(environment variable:
PIP_BUILD_CONSTRAINT)--no-deps
Don’t install package dependencies.
(environment variable:
PIP_NO_DEPS, PIP_NO_DEPENDENCIES)--pre
Include pre-release and development versions. By default, pip only finds stable versions.
(environment variable:
PIP_PRE)-e, --editable <path/url>
Install a project in editable mode (i.e. setuptools “develop mode”) from a local project path or a VCS url.
(environment variable:
PIP_EDITABLE)--src <dir>
Directory to check out editable projects into. The default in a virtualenv is “<venv path>/src”. The default for global installs is “<current dir>/src”.
(environment variable:
PIP_SRC, PIP_SOURCE, PIP_SOURCE_DIR, PIP_SOURCE_DIRECTORY)--ignore-requires-python
Ignore the Requires-Python information.
(environment variable:
PIP_IGNORE_REQUIRES_PYTHON)--no-build-isolation
Disable isolation when building a modern source distribution. Build dependencies specified by PEP 518 must be already installed if this option is used.
(environment variable:
PIP_NO_BUILD_ISOLATION)--check-build-dependencies
Check the build dependencies.
(environment variable:
PIP_CHECK_BUILD_DEPENDENCIES)-C, --config-settings <settings>
Configuration settings to be passed to the build backend. Settings take the form KEY=VALUE. Use multiple --config-settings options to pass multiple keys to the backend.
(environment variable:
PIP_CONFIG_SETTINGS)--no-binary <format_control>
Do not use binary packages. Can be supplied multiple times, and each time adds to the existing value. Accepts either “:all:” to disable all binary packages, “:none:” to empty the set (notice the colons), or one or more package names with commas between them (no colons). Note that some packages are tricky to compile and may fail to install when this option is used on them.
(environment variable:
PIP_NO_BINARY)--only-binary <format_control>
Do not use source packages. Can be supplied multiple times, and each time adds to the existing value. Accepts either “:all:” to disable all source packages, “:none:” to empty the set, or one or more package names with commas between them. Packages without binary distributions will fail to install when this option is used on them.
(environment variable:
PIP_ONLY_BINARY)--prefer-binary
Prefer binary packages over source packages, even if the source packages are newer.
(environment variable:
PIP_PREFER_BINARY)--require-hashes
Require a hash to check each requirement against, for repeatable installs. This option is implied when any package in a requirements file has a --hash option.
(environment variable:
PIP_REQUIRE_HASHES)--progress-bar <progress_bar>
Specify whether the progress bar should be used. In ‘auto’ mode, --quiet will suppress all progress bars. [auto, on, off, raw] (default: auto)
(environment variable:
PIP_PROGRESS_BAR)--group <[path:]group>
Install a named dependency-group from a “pyproject.toml” file. If a path is given, the name of the file must be “pyproject.toml”. Defaults to using “pyproject.toml” in the current directory.
(environment variable:
PIP_GROUP)--no-clean
Don’t clean up build directories.
(environment variable:
PIP_NO_CLEAN)-i, --index-url <url>
Base URL of the Python Package Index (default [https://pypi.org/simple](https://pypi.org/simple)). This should point to a repository compliant with PEP 503 (the simple repository API) or a local directory laid out in the same format.
(environment variable:
PIP_INDEX_URL, PIP_PYPI_URL)--extra-index-url <url>
Extra URLs of package indexes to use in addition to --index-url. Should follow the same rules as --index-url.
(environment variable:
--no-index
Ignore package index (only looking at --find-links URLs instead).
(environment variable:
-f, --find-links <url>
If a URL or path to an html file, then parse for links to archives such as sdist (.tar.gz) or wheel (.whl) files. If a local path or [file://](file:///) URL that’s a directory, then look for archives in the directory listing. Links to VCS project URLs are not supported.
(environment variable:
——————————————————————————————-»»»»»»»»»»»»»»»»»»»»»»»»
(environment variable:
PIP_EXTRA_INDEX_URL)--no-index
Ignore package index (only looking at --find-links URLs instead).
(environment variable:
PIP_NO_INDEX)-f, --find-links <url>
If a URL or path to an html file, then parse for links to archives such as sdist (.tar.gz) or wheel (.whl) files. If a local path or [file://](file:///) URL that’s a directory, then look for archives in the directory listing. Links to VCS project URLs are not supported.
(environment variable:
PIP_FIND_LINKS)——————————————————————————————-»»»»»»»»»»»»»»»»»»»»»»»»