Computer Science and Programming
156K subscribers
446 photos
32 videos
37 files
740 links
Channel specialized for advanced topics of:
* Artificial intelligence,
* Machine Learning,
* Deep Learning,
* Computer Vision,
* Data Science
* Python

Admin: @otchebuch

Memes: @memes_programming

Ads: @Source_Ads,
https://telega.io/c/computer_science
Download Telegram
๐Ÿฎ๐Ÿฌ ๐—ฆ๐—ค๐—Ÿ ๐—พ๐˜‚๐—ฒ๐—ฟ๐˜† ๐—ผ๐—ฝ๐˜๐—ถ๐—บ๐—ถ๐˜‡๐—ฎ๐˜๐—ถ๐—ผ๐—ป ๐˜๐—ฒ๐—ฐ๐—ต๐—ป๐—ถ๐—พ๐˜‚๐—ฒ๐˜€

Below are the SQL query optimization techniques that I found to be significant, listed in the top 20:

1. Create an index on huge tables (>1.000.000) rows
2. Use EXIST() instead of COUNT() to find an element in the table
3. SELECT fields instead of using SELECT *
4. Avoid Subqueries in WHERE Clause
5. Avoid SELECT DISTINCT where possible
6. Use WHERE Clause instead of HAVING
7. Create joins with INNER JOIN (not WHERE)
8. Use LIMIT to sample query results
9. Use UNION ALL instead of UNION wherever possible
10. Use UNION where instead of WHERE ... or ... query.
11. Run your query during off-peak hours
12. Avoid using OR in join queries
14. Choose GROUP BY over window functions
15. Use derived and temporary tables
16. Drop the index before loading bulk data
16. Use materialized views instead of views
17. Avoid != or <> (not equal) operator
18. Minimize the number of subqueries
19. Use INNER join as little as possible when you can get the same output using LEFT/RIGHT join.
20. Frequently try to use temporary sources to retrieve the same dataset.
๐Ÿ‘103
๐—›๐—ผ๐˜„ ๐—ฑ๐—ผ๐—ฒ๐˜€ ๐——๐—ถ๐˜€๐—ฐ๐—ผ๐—ฟ๐—ฑ ๐—ต๐—ฎ๐—ป๐—ฑ๐—น๐—ฒ ๐—ฎ ๐—บ๐—ถ๐—น๐—น๐—ถ๐—ผ๐—ป ๐—ผ๐—ป๐—น๐—ถ๐—ป๐—ฒ ๐˜‚๐˜€๐—ฒ๐—ฟ๐˜€ ๐—ถ๐—ป ๐—ฎ ๐˜€๐—ถ๐—ป๐—ด๐—น๐—ฒ ๐˜€๐—ฒ๐—ฟ๐˜ƒ๐—ฒ๐—ฟ?

As time passed, the overall size of Discord's user base, including its most prominent communities, has grown massively. This affected servers that started to slow down and hit their throughput limits. So, they needed to scale individual Discord servers from tens of thousands to millions of concurrent users.

Whenever someone sends a message on Discord or joins a channel, they need to update the date UI of everyone online on that server. They call that server a "๐—ด๐˜‚๐—ถ๐—น๐—ฑ," which runs in a ๐˜€๐—ถ๐—ป๐—ด๐—น๐—ฒ ๐—˜๐—น๐—ถ๐˜…๐—ถ๐—ฟ ๐—ฝ๐—ฟ๐—ผ๐—ฐ๐—ฒ๐˜€๐˜€, while there is another process (a "๐˜€๐—ฒ๐˜€๐˜€๐—ถ๐—ผ๐—ป") for each connected client. The guild process tracks sessions of users who are members of that guild and are responsible for actions to those sessions. When sessions get updates, forward them to the web socket socket to the client.

The main issue is that ๐—ฎ ๐˜€๐—ถ๐—ป๐—ด๐—น๐—ฒ ๐—บ๐—ฒ๐˜€๐˜€๐—ฎ๐—ด๐—ฒ ๐—ป๐—ฒ๐—ฒ๐—ฑ๐˜€ ๐˜๐—ผ ๐—ด๐—ผ ๐˜๐—ผ ๐˜๐—ต๐—ฒ ๐—ป๐˜‚๐—บ๐—ฏ๐—ฒ๐—ฟ ๐—ผ๐—ณ ๐—ฝ๐—ฒ๐—ผ๐—ฝ๐—น๐—ฒ ๐—ผ๐—ป๐—น๐—ถ๐—ป๐—ฒ on that server, which means if a server has 1000 people online and they all send a message once, that's 1 million notifications.
๐Ÿ‘51๐Ÿ‘Ž1
So, how they solved it? ๐—ง๐—ต๐—ฒ๐˜† ๐—ณ๐—ถ๐—ฟ๐˜€๐˜ ๐˜๐—ฟ๐—ถ๐—ฒ๐—ฑ ๐˜๐—ผ ๐˜‚๐—ป๐—ฑ๐—ฒ๐—ฟ๐˜€๐˜๐—ฎ๐—ป๐—ฑ ๐—ต๐—ผ๐˜„ ๐˜๐—ต๐—ฒ ๐˜€๐˜†๐˜€๐˜๐—ฒ๐—บ ๐—ฝ๐—ฒ๐—ฟ๐—ณ๐—ผ๐—ฟ๐—บ๐˜€. They tracked what Elixir processes were doing, if they were stuck waiting on something, etc. They recorded the event types, how many of each kind of message they received, and their processing times. In addition, they tried to understand how much memory they use, the performances of garbage collectors, etc.

After the analysis, they ๐—ฐ๐—ฟ๐—ฒ๐—ฎ๐˜๐—ฒ๐—ฑ ๐˜๐—ต๐—ฒ ๐—ณ๐—ผ๐—น๐—น๐—ผ๐˜„๐—ถ๐—ป๐—ด ๐˜€๐˜๐—ฟ๐—ฎ๐˜๐—ฒ๐—ด๐˜†:

๐Ÿญ. ๐—ฃ๐—ฎ๐˜€๐˜€๐—ถ๐˜ƒ๐—ฒ ๐˜€๐—ฒ๐˜€๐˜€๐—ถ๐—ผ๐—ป๐˜€: Discord significantly reduced the amount of data processed and sent by differentiating between active and passive user connections, cutting the fanout work by 90% for large servers.

๐Ÿฎ. ๐—ฅ๐—ฒ๐—น๐—ฎ๐˜†๐˜€: Implementing a relay system (read - multithreading) allowed Discord to split the fanout process across multiple machines, enabling a single guild to utilize more resources and support more prominent communities. Relays maintain connections to the sessions instead of the guild and are responsible for doing fanout with permission checks.

๐Ÿฏ. ๐—ช๐—ผ๐—ฟ๐—ธ๐—ฒ๐—ฟ ๐—ฝ๐—ฟ๐—ผ๐—ฐ๐—ฒ๐˜€๐˜€๐—ฒ๐˜€ ๐—ฎ๐—ป๐—ฑ ๐—˜๐—ง๐—ฆ: To maintain server responsiveness, Discord employed worker processes and Erlang Term Storage (ETS) for operations requiring iteration over large sets of members, thus avoiding bottlenecks in the guild process. ETS is an in-memory database that supports the ability of multiple Elixir processes to access it safely. This enables the creation of a new worker process and passes the ETS table so this process can run expensive operations and offload the central guild server.

๐Ÿ”—https://discord.com/blog/maxjourney-pushing-discords-limits-with-a-million-plus-online-users-in-a-single-server
Please open Telegram to view this post
VIEW IN TELEGRAM
๐Ÿ‘47โค1
๐—›๐—ผ๐˜„ ๐˜๐—ผ ๐—ฐ๐—ผ๐—ฑ๐—ฒ ๐˜„๐—ถ๐˜๐—ต ๐—š๐—ถ๐˜๐—›๐˜‚๐—ฏ ๐—–๐—ผ๐—ฝ๐—ถ๐—น๐—ผ๐˜?

A recent study by GitHub and Microsoft discovered that AI now authors 46% of new code. They also found that overall developer productivity surged by 55%, leading to more efficient coding processes. When we talk about AI-powered coding, we mainly talk about GitHub Copilot.

But ๐—ต๐—ผ๐˜„ ๐—š๐—ถ๐˜๐—›๐˜‚๐—ฏ ๐—–๐—ผ๐—ฝ๐—ถ๐—น๐—ผ๐˜ ๐˜„๐—ผ๐—ฟ๐—ธ๐˜€?

The process goes in the following steps:

๐Ÿญ. ๐—ฆ๐—ฒ๐—ฐ๐˜‚๐—ฟ๐—ฒ ๐—ฝ๐—ฟ๐—ผ๐—บ๐—ฝ๐˜ ๐˜๐—ฟ๐—ฎ๐—ป๐˜€๐—บ๐—ถ๐˜€๐˜€๐—ถ๐—ผ๐—ป: Your prompts are securely sent to Copilot, ensuring data privacy.

๐Ÿฎ. ๐—–๐—ผ๐—ป๐˜๐—ฒ๐˜…๐˜๐˜‚๐—ฎ๐—น ๐˜‚๐—ป๐—ฑ๐—ฒ๐—ฟ๐˜€๐˜๐—ฎ๐—ป๐—ฑ๐—ถ๐—ป๐—ด: Copilot analyzes the code around your cursor, the file type, and other open files to offer relevant suggestions.

๐Ÿฏ. ๐—–๐—ผ๐—ป๐˜๐—ฒ๐—ป๐˜ ๐—ณ๐—ถ๐—น๐˜๐—ฒ๐—ฟ๐—ถ๐—ป๐—ด: It filters out personal data and inappropriate content, focusing solely on generating helpful code.

๐Ÿฐ. ๐—–๐—ผ๐—ฑ๐—ฒ ๐—ด๐—ฒ๐—ป๐—ฒ๐—ฟ๐—ฎ๐˜๐—ถ๐—ผ๐—ป: Based on the intent identified in your prompts, Copilot crafts code suggestions that align with your coding style and project standards.

๐Ÿฑ. ๐—จ๐˜€๐—ฒ๐—ฟ ๐—ถ๐—ป๐˜๐—ฒ๐—ฟ๐—ฎ๐—ฐ๐˜๐—ถ๐—ผ๐—ป: Here, we can decide whether to use, tweak, or reject Copilot's suggestions.

๐Ÿฒ. ๐—™๐—ฒ๐—ฒ๐—ฑ๐—ฏ๐—ฎ๐—ฐ๐—ธ ๐—น๐—ผ๐—ผ๐—ฝ: Copilot learns from your interactions, improving its suggestions. Every time you tweak or reject its ideas, he knows from it. It employs techniques like zero-shot (asking without examples), one-shot (asking with an example), and few-shot learning (providing multiple examples) to adapt to our instructions, whether you provide examples or not.

๐Ÿณ. ๐—ฃ๐—ฟ๐—ผ๐—บ๐—ฝ๐˜ ๐—ต๐—ถ๐˜€๐˜๐—ผ๐—ฟ๐˜† ๐—ฟ๐—ฒ๐˜๐—ฒ๐—ป๐˜๐—ถ๐—ผ๐—ป: It remembers past prompts and interactions, making future suggestions more accurate.
๐Ÿ‘74๐Ÿ‘Ž10โค1
Please open Telegram to view this post
VIEW IN TELEGRAM
๐Ÿ‘41โค1
Use KeePassXC to sign your git commits

๐Ÿ”— https://code.mendhak.com/keepassxc-sign-git-commit-with-ssh/
Please open Telegram to view this post
VIEW IN TELEGRAM
๐Ÿ‘40
A library for building fast, reliable and evolvable network services

๐Ÿ’ป https://github.com/cloudflare/pingora/tree/main
Please open Telegram to view this post
VIEW IN TELEGRAM
๐Ÿ‘45โค1
This media is not supported in your browser
VIEW IN TELEGRAM
Direct-a-Video: driving Video Generation

Direct-a-Video is a text-to-video generation framework that allows users to individually or jointly control the camera movement and/or object motion. Authors: City University of HK, Kuaishou Tech & Tianjin.

๐‡๐ข๐ ๐ก๐ฅ๐ข๐ ๐ก๐ญ๐ฌ:
โœ… Decoupling camera/object motion in gen-AI
โœ… Allowing users to independently/jointly control
โœ… Novel temporal cross-attention for cam motion
โœ… Training-free spatial cross-attention for objects
โœ… Driving object generation via bounding boxes

Paper: https://arxiv.org/pdf/2402.03162.pdf
Project: https://direct-a-video.github.io/
Please open Telegram to view this post
VIEW IN TELEGRAM
๐Ÿ‘34๐Ÿ‘Ž3โค1
๐Ÿ–ฅ Searchable table in Python using Flet

This tutorial will show you how to create an interactive table using Flet.
Moreover, with search and filtering functions, which is very cool ๐Ÿ”ฅ

๐Ÿ”œ Step by step tutorial
Please open Telegram to view this post
VIEW IN TELEGRAM
Please open Telegram to view this post
VIEW IN TELEGRAM
๐Ÿ‘39
Harvard CS50 โ€“ Free Computer Science Course (2023 Edition)

Here are the lectures included in this course:

Lecture 0 - Scratch
Lecture 1 - C
Lecture 2 - Arrays
Lecture 3 - Algorithms
Lecture 4 - Memory
Lecture 5 - Data Structures
Lecture 6 - Python
Lecture 7 - SQL
Lecture 8 - HTML, CSS, JavaScript
Lecture 9 - Flask
Lecture 10 - Emoji
Cybersecurity

https://www.freecodecamp.org/news/harvard-university-cs50-computer-science-course-2023/
๐Ÿ‘166๐Ÿ‘Ž5โค1๐Ÿ˜1
Amber - the programming language compiled to Bash

๏ฟผhttps://amber-lang.com/ https://amber-lang.com/
๐Ÿ‘63
What's New in Kotlin 2.0.0

๐Ÿ”— https://kotlinlang.org/docs/whatsnew20.html
Please open Telegram to view this post
VIEW IN TELEGRAM
๐Ÿ‘33๐Ÿ—ฟ1
Uber Migrates 1 Trillion Records from DynamoDB to LedgerStore to Save $6 Million Annually

๐Ÿ”— https://www.infoq.com/news/2024/05/uber-dynamodb-ledgerstore/
Please open Telegram to view this post
VIEW IN TELEGRAM
๐Ÿ‘39
Please open Telegram to view this post
VIEW IN TELEGRAM
๐Ÿ‘14
SQL at 50: Whatโ€™s next for the structured query language?

๐Ÿ”— https://www.infoworld.com/article/3715453/sql-at-50-whats-next-for-the-structured-query-language.html
Please open Telegram to view this post
VIEW IN TELEGRAM
๐Ÿ‘26
It's time for Stack Overflow's annual Developer Survey! From your preferred programming languages, the new tech you want to explore, and much, much more, we're calling all who code to weigh in.

Take the survey: https://stackoverflow.az1.qualtrics.com/jfe/form/SV_6rJVT6XXsfTo1JI?utm_medium=social&utm_source=twitter&utm_campaign=dev-survey-2024&utm_content=
๐Ÿ‘40๐Ÿ‘Ž3โค1
#bugs

Github CoPilot Re-Enables Itself After Being Disabled

๐Ÿ”— https://github.com/microsoft/vscode-copilot-release/issues/1248
Please open Telegram to view this post
VIEW IN TELEGRAM
๐Ÿ‘36
What's New in C# 13: Enhanced Params, Performance Boosts, and New Extension Types

๐Ÿ”— https://www.infoq.com/news/2024/05/csharp-13-preview/
Please open Telegram to view this post
VIEW IN TELEGRAM
๐Ÿ‘36
40 Tips From A Senior to a Junior developer.

Part I:

1. Master the Basics: Ensure you have a strong grasp of fundamental programming concepts.
2. Write Readable Code: Prioritize readability over cleverness.
3. Use Version Control: Learn Git and make frequent commits with meaningful messages.
4. Understand Your Tools: Get familiar with your IDE, debugger, and terminal.
5. Comment Wisely: Use comments to explain why, not what.
6. Practice Problem-Solving: Regularly work on coding challenges to improve your skills.
7. Learn Design Patterns: Study common design patterns and their use cases.
8. Ask Questions: Donโ€™t hesitate to seek help when stuck.
9. Read Documentation: Thoroughly read the documentation for libraries and frameworks you use.
10. Write Tests: Develop a habit of writing unit and integration tests.
11. Refactor Regularly: Continuously improve your codebase.
12. Learn Debugging: Master the art of debugging to solve issues efficiently.
13. Understand Algorithms: Study common algorithms and their complexities.
14. Keep Learning: Stay updated with new technologies and industry trends.
15. Build Projects: Work on side projects to apply what you learn.
16. Use Code Reviews: Participate in code reviews to learn from others and improve your code.
17. Learn SQL: Know how to work with databases and write efficient queries.
18. Learn about the business domain youโ€™re working in.
19. Build relationships with other developers.
20. Stay Organized: Use task management tools to keep track of your work.
๐Ÿ‘171โค3๐Ÿ‘Ž3