GrabLab OSINT News
149 subscribers
15 photos
13 links
News from @TgScanRobot project.
Chat room: @grablab_osint
Download Telegram
to view and join the conversation
Project progress for second half of week:
* I have spent insane amount of time on testing MongoDB cluster. I have managed stable work of primary and secondary MongoDB servers under the load of 50-100k updates per second.
* More database updates. Current numbers: 1.4 billion user-group connections, 220 million unique users, 1.1 million groups.
* @ChatSearchRobot is alive again! It is paid now, one query costs 0.05 USD. The search result contains now 20 groups (was 5).
Now TgScanRobot display extra information in search result (if it has such information):
* history of @usernames used with found ID
* history of IDs which were connected to found @username
Now each search result of TgScanRobot has attached CSV file which contains data in CSV form. Each line of CSV file is formatted as "group-numeric-id;group-username;group-title;date"
Project progress for first half of week:
* @ChatSearchRobot now can accept multiple groups in search query. It does not mean it will provide separate results for each group. It will show groups similar to the union of groups provided in request.
* Simple info page for @ChatSearchRobot: https://tgdev.io/chatsearch
* More database updates, there are now 1481 million members, 226 million users, 1.1 million groups
* @TgScanRobot now attaches CSV file to search result. Inside the file there are just same groups from search result formatted as CSV data
* @TgScanRobot now shows extra info: history of usernames connected to requested Telegram ID and history of Telegram IDs connected to requested username
At this moment there are 1 million 135 thousand groups in database of TgScanRobot. Would you be interested in searching groups in this database? It is possible to search groups by keyword in a group's title or by keyword in a group's username. If you are interested, please write what reasonable cost you see for such service i.e. the cost for 1 group. The total cost of search operation will be a cost for 1 group multiplied by a number of found groups.
Project progress by second half of week:

1) I have connected new payment method: QIWI.

2) More database updates. Now it contains 234m groups, 1.19m unique users and 1.58 billion user-group connections.

3) I have implemented bulk query feature. Now you can upload text file with multiple queries and get result as JSON or CSV files. This feature is in testing mode yet. During tests there is 30% discount on bulk query results which returned groups (no discount on results which returned only basic profile data). You can use use bulk interface for single queries too, just upload a file with one query. A page to work with bulk queries is here: https://tgdev.io/tgscan/bulk
New feature on Bulk Query results page. Now you can export result in custom format.
Now you can send a joinchat link to TgScanRobot and it will show you ID of Telegrm user which created this link. It does not work for all links, only for old ones, created more than 5 years ago — when the Telegram was including in the link the ID of Telegram user which created a link.

You can read more about such links in this article https://telegra.ph/How-to-find-the-owner-and-administrator-of-a-Telegram-channel-11-02 (Section 2, "Private link").

Example of such link: https://t.me/joinchat/FNPR30Fs51OX1853MfCKVg. If you send this link to the bot TgScanRobot it will show you ID of user which created this link. The paid version of TgScanRobot will also search data for this user and show you result.
New bulk query feature. Now you do not need to convert CSV into excel. You can download excel files directly.
Project progress by first half of week:
1) An option to setup custom format of bulk query result (https://t.me/grablab_osint_news/24)
2) Detection user ID in joinchat link (https://t.me/grablab_osint_news/25)
3) An option to download bulk query result as excel file (https://t.me/grablab_osint_news/26)
4) Database update. Current numbers are 1.2 million groups, 239 million unique users, 1.6 billion user-group connections.
Weekly progress report:
1) I have done no new features but I was walking a lot in the forest. I think it is good for the project, in long term perspective.
2) I have started researching new ways of finding new groups. Right now I am testing the collecting groups by sending queries to Telegram built-in search feature.
3) As usual I was re-scanning every day every known group. Current numbers are: 1.3 million groups, 255 million unique users, 1.8 billion relations user-group.
4) I am thinking about crawling history of messages from each known group. This way I will greatly extend historical quality of my data.
Telegram Account Identificator

In Telegram every user is identified by a unique numeric ID. Also a user might have @username, or might not, or a user might change @username. The numeric ID could not be changed, it always exists and it is the same from the moment of creating a Telegram account.

How to find out the ID of a user? There are multiple method.

First, the most easy method. If you have a conversation with a user or you see their messages in a public group then you forward their message to bot @TgScanRobot. The bot will show your ID of the user. This method does not always work. A user might change their Telegram settings and restrict other people to get information from messages of the user.

Second method works if a user is in your contact list or if he/she is a member of group which you are a member of too. Using particular Telegram applications you need to look at their profile — there will be ID displayed. Official Telegram applications do not show user's ID. You have to use non-official applications.

Applications which show user’s ID:


* Kotatogram (Windows, Linux, MacOs) kotatogram.github.io
* iMe Messenger (Android, iOS) imem.app
* BGram (Android) play.google.com/store/apps/details?id=org.telegram.BifToGram
* Graph Messenger (Android) play.google.com/store/apps/details?id=ir.ilmili.telegraph
* Web application web.telegram.org — you can get a user’s ID from an URL in a browser address bar when you are looking at a user’s profile. Example of such URL: https://web.telegram.org/z/#1557162396

Third method. Use it when you can’t see a user’s profile but you know the phone number of the user. In the Telegram application add a new contact using the phone number of the user. If the user is registered in Telegram a new contact will appear in your address book and you’ll be able to see their profile. How to get Telegram ID using the user’s profile is described in the second method (see above).

How can you use a Telegram user’s ID? At least you can use it for pointing to the user in conversation with other people. Also you can use it in work with informational services. For example, if you send user’s ID to bot @TgScanRobot you'll get list of groups which this user is a member of.
Finally! Two billion user-group connections in database, also the number of known groups reached 1.5 million :)
One more method I have started using for finding new groups. I send requests to standard Telegram search and remember found groups. For the start, I took @username of already known groups and split each @username into pieces using "_" as delimiter. I got about one million words. I searched these words in Telegram search and found many new groups, as you can see on screenshot. Next step is to process city names. From geonames.org I have downloaded the list of cities with a population more than 500 people. That list contains city names in English language and also translations to other languages. I have split these names into pieces using space and dash as delimiters. I got about one million words. I will search these words and I hope to find many new groups 🌚 If you have ideas about what else list of words is to search, please write in comments.
Dataset of @TgScanRobot has reached 300m unique users and 2.5 billiard group members 🌚
Not all users of TgScanRobot are aware about other useful bot which utilizes the TgScanRobot's database. I am talking about @ChatSearchRobot. This bot can show you groups related to the group you've sent to the bot. The relativity is based on number of same members in two groups.

Possible use cases for the bot:
* find new groups for advertising
* find new places to ask your question
* research what people talk about in specific groups

The bot is also paid. You check the price at https://tgdev.io/chatsearch The bot uses very same credits as the TgScanRobot uses.