implemented. URLhausURLhaus is a prominent web service tracking malicious URLs. The web service also provides exports containing new detected URLs. Those malware URLs serve as perfect input to our crawler as we mainly want to analyze malicious domains. Recently submitted URLS are retrieved and search results are not refined through the API request (i.e. through tags or other parameters available). The HTTP request made in this input module to the URLHaus API can be modifed to further refine the results obtained. PhishTankPhishTank is a website that collects phishing URLs. Users have the possibility to submit new found phishing pages. An export with active phishing URLs can be generated and downloaded from this web service via API. So this is also an ideal collection for our crawler. Processing ModulesSubCrawl comes with several processing modules. The processing modules all follow similar behavior on how they provide results back to the core engine. If matches are found, results are returned to the core engine and later provided to the storage modules. Below is a list of processing modules. SDHashThe SDHash processing modue is used to calculate a similarity hash of the HTTP response. The minimum size of the content must is 512 bytes to be able to successfully calculate a hash. This is probably the most complicated processing module to install, as it requires Protobuf and depending on the target host it must be recompiled. Therefore this processing module is deactivated by default. An already compiled version can be found in crawler/processing/minisdhash/ which requires protobuf-2.5.0 and python3.6. Those binaries were compiled on an Ubuntu 18.04.5 LTS x64. Following the installation instructions: # Protobuf installation
> apt-get update
> apt-get -y install libssl-dev libevent-pthreads-2.1-6 libomp-dev g++
> apt-get -y install autoconf automake libtool curl make g++ unzip
> wget https://github.com/protocolbuffers/protobuf/releases/download/v2.5.0/protobuf-2.5.0.zip
> unzip protobuf-2.5.0.zip
> cd protobuf-2.5.0
> ./configure
> make
> sudo make install

# Python3.6 installation
> apt-get install python3.6-dev
> sudo ldconfig

# SDHash installation
> git clone https://github.com/sdhash/sdhash.git
> cd sdhash
> make
> make install
> ldconfig JARMJARM is a tool that fingerprints TLS connections developed by Salesforce. The JARM processing module performs a scan of the domain and returns a JARM hash with the domain to the core engine. Depending on the configuration of a web server, the TLS handshake has different properties. By calculating a hash of the attributes of this handshake, these differences can be used to track web server configurations. TLSHThe TLSH processing module is similar to the SDHash processing module used to calculate a similarity hash. The advantage of the TLSH is, that the installation is much simpler and the input minium is smaller with 50 bytes. As most webshell logins are rather small and were the focus of our research, we activated this processing module by default. YARAThe YARA processing module is used to scan HTTP response content with YARA rules. To invoke this processing module, provide the value YARAProcessing as a processing module argument. For example, the following command will load the YARA processing module and produce output to the console via the ConsoleStorage storage module. python3 subcrawl.py -p YARAProcessing -s ConsoleStorage Currently, the YARA processing module is used to identify webshell logins and various other interesting content. YARA rules included with this project:

* protected_webshell: Identifies login pages of password-protected webshells
* js_webshell_tracking_script: Identifies backdoored plugins/themes that use JavaScript to notifies the attacker when the webshell becomes active
* open_webshell: Identifies open webshells (i.e. webshells that are not protected vi[...]

a login)
* php_webshell_backend: Identifies PHP webshell backend used by the attacker

Sample output: https://blogger.googleusercontent.com/img/a/AVvXsEiClmCCGwqGkrCWz_nPrcl3evUAFsk_iMyg6HfaGM7tkOnYh7CR6kem4ji2RipfXEBVl5tK1Oj1qFD_RXg2zk1bJDbdqT3k0AsElMVCy-4vBUsWLdm7YosI5T5GV4jsxl7RlGkvJcVidf3aoeP196dQm6LE4HuWtYi4nzMQWsEMwpGu3L1PG78VFApdWQ=w640-h68 To add additional YARA rules, you can add .YAR files to the yara-rules folder, and then include the rule file by adding an include statement to combined-rules.yar. ClamAVThe ClamAV processing module is used to scan HTTP response content during scanning with ClamAV. If a match is found, it is provided to the various output modules. To invoke this processing module, provide the value ClamAVProcessing as a processing module argument. For example, the following command will load the ClamAV processing module and produce output to the console via the ConsoleStorage storage module. python3 subcrawl.py -p ClamAVProcessing -s ConsoleStorage Sample output: https://blogger.googleusercontent.com/img/a/AVvXsEg50CLwfXVjNoWzeB1kFimGQZWqih_SCVMxpE2rzGXwY_4v_Rom2t9825rxXlo9NqOSViIbhKg8bnQ3WVo_A7blPFJkUMHzauEmNVGHf1DG9j1WtHt9ivFenqDDuWpEY8BQ8uqoIhLl3AUzQqnTqVGJKnWA90I7dEB2FU_2rOxhOVLtHrImlrUwpgWDXA=w640-h168 To utilize this module, ClamAV must be installed. From a terminal, install ClamAV using the APT package manager: $ sudo apt-get install clamav-daemon clamav-freshclam clamav-unofficial-sigs Once installed, the ClamAV update service should already be running. However, if you want to manually update using freshclam, ensure that the service is stopped: sudo systemctl stop clamav-freshclam.service And then run freshclam manually: $ sudo freshclam Finally, check the status of the ClamAV service: $ sudo systemctl status clamav-daemon.service If the service is not running, you can use systemctl to start it: $ sudo systemctl start clamav-daemon.service PayloadThe Payload processing module is used to identify HTTP response content using the libmagic library. Additionally, SubCrawl can be configured to save content of interest, such as PE files or archives. To invoke this processing module, provide the value PayloadProcessing as a processing module argument. For example, the following command will load the Payload processing module and produce output to the console: python3 subcrawl.py -p PayloadProcessing -s ConsoleStorage There are no additional dependencies for this module.

Sample output: https://blogger.googleusercontent.com/img/a/AVvXsEilskO1_9hJRyrr84vTCCRG91wO_nQOdp5Rw-004LNRX-AH2VFZlDGVcl7Ua_F8QID6ioNGNF263LcWPq7dXWXCilmihZtkfVZ2geUlihFBjnbf_hMFohBPbE9R4cw3RToj-AoKgzvFQExHOOQ-gSH_UbH1GBAFF2kS7folo0VJwbp9y5JYhjF93Uo_zQ=w640-h80 Storage ModulesStorage modules are called by the SubCrawl engine after all URLs from the queue have been scanned. They were designed with two objectives in mind. First, to obtain the results from scanning immediately after finishing the scan queue and secondly to enable long-term storage and analysis. Therefore we not only implemented a ConsoleStorage module but also an integration for MISP and an SQLite storage module. ConsoleTo quickly analyse results directly after scanning URLs, a well-formatted output is printed to the console. This output is best suited for when SubCrawl is used in run-once mode. While this approach worked well for scanning single domains or generating quick output, it is unwieldy for long-term research and analysis. https://blogger.googleusercontent.com/img/a/AVvXsEhJkhXFNc1i5Xqr8YV5JuZ2HENnsCgsgZswwr_lGLhwr_gko7B01zDo6mAycByUKT_vJgFgodVLndDqceLsV5mVBPliLo_KPutJRqxc5alPVdt0LPRjVHFS_LjSVQGZXvya9S8qzwaOHiDz8aMG03NJY6JAmPZFe0fMXlb-p30HW1PZPNrOUWlhfsbjgg=w640-h146 SQLiteSince the installation and configuration of MISP can be time-consuming, we implemented another module which stores the data in an SQLite database. To[...]

present the data to the user as simply and clearly as possible, we also developed a simple web GUI. Using this web application, the scanned domains and URLs can be viewed and searched with all their attributes. Since this is only an early version, no complex comparison features have been implemented yet. https://blogger.googleusercontent.com/img/a/AVvXsEi1mKlPJyBw7H2hMCOsltdf_iGvs2otlfMg9hV0AulAz3_A0vrtONKdxZJ_xZIXCKQjgiB9Tr4hCPmH25Szqoo373peOGMK9e6oJIiwu_SqaCE3vv0NVZZTxNRvTnpuFC4qo0yVxglLURZxW5-lYYJT9lxHlwG2MwYcykcZROuOnTXonBT-fc4krSphAg=w640-h192 MISPMISP is an open-source Intelligence Platform" href="https://www.kitploit.com/search/label/Threat%20Intelligence%20Platform">threat intelligence platform with a flexible data model and API to store and analyze threat data. SubCrawl stores crawled data in MISP events, publishing one event per domain and adding any identified open directories as attributes. MISP also allows users to define tags for events and attributes. This is helpful for event comparison and link analyses. Since this was one of our primary research goals, we enriched the data from URLHaus when exporting SubCrawl’s output to MISP. URLHaus annotates its data using tags which can be used to identify a malware family or threat actor associated with a URL. For each open directory URL, the module queries locally-stored URLHaus data and adds URLHaus tags to the MISP event if they match. To avoid having a collection of unrelated attributes for each MISP event, we created a new MISP object for scanned URLs, called opendir-url. This ensures that related attributes are kept together, making it easier to get an overview of the data. https://blogger.googleusercontent.com/img/a/AVvXsEiPrAeBa7PFkTTI-acwcy-Gk8oDIX8tuleYKyQHx3Vk0eqy6EyATm9xJlf_KSWyTFmhIB0GPGnAdhd-6Ed1xZniJxfKEVhLIdY11-ouCkv_sNfDIE0Velmz-KP0rVeTwthh1Jyd6ZHdGdNL-HDG_O2-76xVaYaUJ3jo9E29N7lJVHGGbRPgUPZMcnd8IQ=w640-h278 Building your own ModulesTemplates for processing and storage modules are provided as part of the framework. Processing ModulesProcessing modules can be found under crawler->processingand a sample module file example_processing.pyfound in this directory. The template provides the necessary inheritance and imports to ensure execution by the framework. The init function provides for module initialization and receives an instance of the logger and the global configuration. The logger is used to provide logging information from the processing modules, as well as throughout the framework.

The process function is implemented to process each HTTP response. To this end, it receives the URL and the raw response content. This is where the work of the module is implemented. This function should return a dictionary with the following fields:

* hash: the sha256 of the content
* url: the URL the content was retrieved from
* matches: any matching results in the module, For example, libmagic or YARA results.

A unique class name must be defined and is used to define this module when including it via the -p argument or as a default processing module in the configuration file.

Finally, add an import statement in __init__.py, using your class name: from .<replace_processing import <replaceProcessing Storage ModulesStorage modules can be found under crawler->storageand a sample module file example_storage.pyfound in this directory. Similar to the processing modules, init function provides for module initialization and receives an instance of the logger and the global configuration. The store_results function receives structured data from the engine at intervals defined by the batch size in the configuration file.

A unique class name must be defined and is used to load the module when including it via the -s argument or as a default processing module in the configuration file. Presentations and Other Resources2021:

* BlackHat[...]

Arsenal USA
* VirusBulletin Localhost - Upcoming LicenseSubCrawl is licensed under the MIT license Download Subcrawl

Dark Reading: Attacks/Breaches
Ransomware Rise Pushes Organizations to Prepare for Attack

Ransomware attacks continue to grow in number and severity, data shows, but organizations are stepping up to prepare for the threat.

hacking: security in practice
This group has ended (whatsapp)

"{d1StG69zitNHoovr3QiHaKm0H+CzJIHRloYi60vEMCk=}" [ 1, 2, 3; 2, 4, 5; 2, 2, 2; 8, 3, 3 ]

Wtf even is this ?

submitted by /u/peru_nub
[link] [comments]

Hacking on Medium
The Sheikh, the Businessman and a Hacking Thriller on 3 Continents

https://cdn-images-1.medium.com/max/1050/0*3eXypNbRqOm8xzKT.jpg
Although the decide questioned the credibility of that story, his ruling towards Mr.

Continue reading on Medium »

Hacking on Medium
How i got Swag from Sony ?

https://cdn-images-1.medium.com/max/2160/1*ZuVO145_wk-joa7V4WwvYQ.jpeg
All i can say it was accidental bug i found (low hanging fruit)

Continue reading on Medium »

Hacking on Medium
REvil group taken down by the Feds

https://cdn-images-1.medium.com/max/1280/1*dSh7axd5LitXVhq3ZWo_rQ.jpeg
On Oct 21 (Reuters) said  — The ransomware group REvil was itself hacked and forced offline this week by a multi-country operation, And…

Continue reading on Medium »

Hacking on Medium
You should start using a password manager right now

https://cdn-images-1.medium.com/max/1200/1*7zUj4HGAHGKUBaSsYaNCQQ.jpeg
It’s morning. You’re scrolling through Twitter or just any news feed of your choice and you see the following headline: “MILLIONS OF…

Continue reading on Medium »

Hacking on Medium
What is Termux ?

Termux is an android terminal emulator that allows for running a Linux Environment on a Android device which work directly without root or…

Continue reading on Medium »

Hacking on Medium
Understanding Credentials Stuffing and Password Spraying

https://cdn-images-1.medium.com/max/1600/0*yYPDFzoX58J1Nilz
Introduction

Continue reading on Medium »

Solana Bug Bounty Hunting with Soteria
https://medium.com/@greencore.xyz/solana-bug-bounty-hunting-with-soteria-1b8b75c741de?source=rss------bug_bounty-5

Recently, using Soteria we identified a vulnerability in an on-chain Solana smart contract (jet-v1) and have been awarded a bug bounty…Continue reading on Medium » (https://medium.com/@greencore.xyz/solana-bug-bounty-hunting-with-soteria-1b8b75c741de?source=rss------bug_bounty-5)