Black Hat Ethical Hacking
Microsoft is disabling Excel 4.0 macros by default to protect users
___________________________
@hacking_Attack
@Hacking_Video
Microsoft is disabling Excel 4.0 macros by default to protect users
___________________________
@hacking_Attack
@Hacking_Video
Black Hat Ethical Hacking
Microsoft is disabling Excel 4.0 macros by default to protect users | Black Hat Ethical Hacking
Microsoft will soon begin disabling Excel 4.0 XLM macros by default in Microsoft 365 tenants to protect customers from malicious documents.
Covert-Tube - Youtube As Covert-Channel - Control Systems Remotely And Execute Commands By Uploading Videos To Youtube
A program to control systems remotely by uploading videos to Youtube using Python to create the videos and the listener, emulating some malware I was reading about. It allows to create videos with frames formed of simple text, QR codes with cleartext or QR codes using AES encryption.Create a videoThe videos can be created using generate_video.py: enter the commands and generate the video writing "exit". The video generated is called by default output.avi (can be updated in config.py):python3 generate_video.py Run the listener and upload the video to Youtubepython3 main.pyThe listener will check the Youtube channel every 300 seconds by default (can be updated in config.py). First the video is uploaded:After finding there is a new video in the channel, it is downloaded and the commands are executed: We can see the output from the commands:ConfigurationUpdate the config.py file:channel_id (Mandatory!!!): Get your Youtube channel ID from here.api_key (Mandatory!!!): To get the API key create an application and generate the key from here.image_type (Optional. Default: "qraes"): Different types of images for the video."cleartext" creates images with the words of the commands."qr" creates QR codes with the commands."qr\aes" creates QR codes with the commands encrypted with AES.upload_seconds_delay (Optional. Default: 300): Seconds delay until checking if a new video has been uploaded.debug (Optional. Default: True): Print messages or not.aes_key (Optional. Default: "covert-tube2021"): Key for AES encryption, used in the "qr\aes" option.generated_video_path (Optional. Default: "output.avi"): Path of video generated with generate_video.py.downloaded_video_path (Optional. Default: "/tmp/test.mp4"): Path where the new video will be downloaded.temp_folder (Optional. Default: "/tmp/"): Path where images of every frame from the video are stored, with the format image_X.png.Installing dependenciesFor all the project:pip3 install Pillow opencv-python youtube_dl pytesseract pyqrcode pypng pyzbar pycryptoFor only the "cleartext" option (OCR):pip3 install Pillow opencv-python youtube_dl pytesseractFor only the QR without encryption option:pip3 install Pillow opencv-python youtube_dl pyqrcode pypng pyzbarFor only the QR with AES encryption option:pip3 install Pillow opencv-python youtube_dl pyqrcode pypng pyzbar pycryptoCreating a standalone binarypyinstaller --onefile main.pycp dist/main covert-tuberm -rf dist buildrm main.specMotivationLately I have been reading about malware using Youtube for controlling their setting remotely. For example, Casbaneiro abuses YouTube to store its C&C server domains. Each video on the channels used by the threat actor contains a description and at the end of these there is a link to a bogus Facebook or Instagram url containing the C&C server domain (Welivesecurity blog). A second example is Numando, which abuses it by encrypting the data in the title of the Youtube videos (other Welivesecurity blog).Knowing this I decided to create a PoC to test the control of remote systems uploading videos to Youtube but, instead of using the title or the description, using the content of the video. It allows to execute any command, but it could be used to change some settings remotely. So this is just a PoC, use it for educational purposes!Download Covert-Tube
Read more...
___________________________
@hacking_Attack
@Hacking_Video
A program to control systems remotely by uploading videos to Youtube using Python to create the videos and the listener, emulating some malware I was reading about. It allows to create videos with frames formed of simple text, QR codes with cleartext or QR codes using AES encryption.Create a videoThe videos can be created using generate_video.py: enter the commands and generate the video writing "exit". The video generated is called by default output.avi (can be updated in config.py):python3 generate_video.py Run the listener and upload the video to Youtubepython3 main.pyThe listener will check the Youtube channel every 300 seconds by default (can be updated in config.py). First the video is uploaded:After finding there is a new video in the channel, it is downloaded and the commands are executed: We can see the output from the commands:ConfigurationUpdate the config.py file:channel_id (Mandatory!!!): Get your Youtube channel ID from here.api_key (Mandatory!!!): To get the API key create an application and generate the key from here.image_type (Optional. Default: "qraes"): Different types of images for the video."cleartext" creates images with the words of the commands."qr" creates QR codes with the commands."qr\aes" creates QR codes with the commands encrypted with AES.upload_seconds_delay (Optional. Default: 300): Seconds delay until checking if a new video has been uploaded.debug (Optional. Default: True): Print messages or not.aes_key (Optional. Default: "covert-tube2021"): Key for AES encryption, used in the "qr\aes" option.generated_video_path (Optional. Default: "output.avi"): Path of video generated with generate_video.py.downloaded_video_path (Optional. Default: "/tmp/test.mp4"): Path where the new video will be downloaded.temp_folder (Optional. Default: "/tmp/"): Path where images of every frame from the video are stored, with the format image_X.png.Installing dependenciesFor all the project:pip3 install Pillow opencv-python youtube_dl pytesseract pyqrcode pypng pyzbar pycryptoFor only the "cleartext" option (OCR):pip3 install Pillow opencv-python youtube_dl pytesseractFor only the QR without encryption option:pip3 install Pillow opencv-python youtube_dl pyqrcode pypng pyzbarFor only the QR with AES encryption option:pip3 install Pillow opencv-python youtube_dl pyqrcode pypng pyzbar pycryptoCreating a standalone binarypyinstaller --onefile main.pycp dist/main covert-tuberm -rf dist buildrm main.specMotivationLately I have been reading about malware using Youtube for controlling their setting remotely. For example, Casbaneiro abuses YouTube to store its C&C server domains. Each video on the channels used by the threat actor contains a description and at the end of these there is a link to a bogus Facebook or Instagram url containing the C&C server domain (Welivesecurity blog). A second example is Numando, which abuses it by encrypting the data in the title of the Youtube videos (other Welivesecurity blog).Knowing this I decided to create a PoC to test the control of remote systems uploading videos to Youtube but, instead of using the title or the description, using the content of the video. It allows to execute any command, but it could be used to change some settings remotely. So this is just a PoC, use it for educational purposes!Download Covert-Tube
Read more...
___________________________
@hacking_Attack
@Hacking_Video
Hacking Articles Tips Tricks Videos Tutorials
Photo
KitPloit - PenTest Tools!
Covert-Tube - Youtube As Covert-Channel - Control Systems Remotely And Execute Commands By Uploading Videos To Youtube
https://1.bp.blogspot.com/-Tkwipt38RVY/YV4Lk3mfEuI/AAAAAAAAwJ4/iSiDlOQuGBUP_O4sdkNdbPG4_-ymCbyhwCNcBGAsYHQ/w640-h108/covert-tube_1_image1.png A program to control systems remotely by uploading videos to Youtube using Python to create the videos and the listener, emulating some malware I was reading about. It allows to create videos with frames formed of simple text, QR codes with cleartext or QR codes using AES encryption. Create a videoThe videos can be created using generate_video.py: enter the commands and generate the video writing "exit". The video generated is called by default output.avi (can be updated in config.py):
*
channel_id (Mandatory!!!): Get your Youtube channel ID from here.
*
api_key (Mandatory!!!): To get the API key create an application and generate the key from here.
*
image_type (Optional. Default: "qr_aes"): Different types of images for the video.
* "cleartext" creates images with the words of the commands.
* "qr" creates QR codes with the commands.
* "qr_aes" creates QR codes with the commands encrypted with AES.
*
upload_seconds_delay (Optional. Default: 300): Seconds delay until checking if a new video has been uploaded.
*
debug (Optional. Default: True): Print messages or not.
*
aes_key (Optional. Default: "covert-tube_2021"): Key for AES encryption, used in the "qr_aes" option.
*
generated_video_path (Optional. Default: "output.avi"): Path of video generated with generate_video.py.
*
downloaded_video_path (Optional. Default: "/tmp/test.mp4"): Path where the new video will be downloaded.
*
temp_folder (Optional. Default: "/tmp/"): Path where images of every frame from the video are stored, with the format image_X.png. Installing dependenciesFor all the project:
___________________________
@hacking_Attack
@Hacking_Video
Covert-Tube - Youtube As Covert-Channel - Control Systems Remotely And Execute Commands By Uploading Videos To Youtube
https://1.bp.blogspot.com/-Tkwipt38RVY/YV4Lk3mfEuI/AAAAAAAAwJ4/iSiDlOQuGBUP_O4sdkNdbPG4_-ymCbyhwCNcBGAsYHQ/w640-h108/covert-tube_1_image1.png A program to control systems remotely by uploading videos to Youtube using Python to create the videos and the listener, emulating some malware I was reading about. It allows to create videos with frames formed of simple text, QR codes with cleartext or QR codes using AES encryption. Create a videoThe videos can be created using generate_video.py: enter the commands and generate the video writing "exit". The video generated is called by default output.avi (can be updated in config.py):
python3 generate_video.py https://1.bp.blogspot.com/-Tkwipt38RVY/YV4Lk3mfEuI/AAAAAAAAwJ4/iSiDlOQuGBUP_O4sdkNdbPG4_-ymCbyhwCNcBGAsYHQ/w640-h108/covert-tube_1_image1.png Run the listener and upload the video to Youtubepython3 main.py The listener will check the Youtube channel every 300 seconds by default (can be updated in config.py). First the video is uploaded: https://1.bp.blogspot.com/-JUQNhQB4kr8/YV4L5rNxQGI/AAAAAAAAwKA/T5vDhdRWlZc2yvdbjh7STJ_Z6NHgKm6CgCNcBGAsYHQ/w640-h540/covert-tube_2_image2.png After finding there is a new video in the channel, it is downloaded and the commands are executed: https://1.bp.blogspot.com/-dMmLbwkGoMI/YV4L_vIrZFI/AAAAAAAAwKE/jpMwYwENYnYXm-COA4tPzQlDmBFYYw60ACNcBGAsYHQ/w640-h184/covert-tube_3_image3.png We can see the output from the commands: https://1.bp.blogspot.com/-BlFFwEUZSoA/YV4MEF8ZsAI/AAAAAAAAwKI/XWyhSpS7k4E9jMu12cI5f2oqsgTt6dr5ACNcBGAsYHQ/w640-h348/covert-tube_4_image4.png ConfigurationUpdate the config.py file:*
channel_id (Mandatory!!!): Get your Youtube channel ID from here.
*
api_key (Mandatory!!!): To get the API key create an application and generate the key from here.
*
image_type (Optional. Default: "qr_aes"): Different types of images for the video.
* "cleartext" creates images with the words of the commands.
* "qr" creates QR codes with the commands.
* "qr_aes" creates QR codes with the commands encrypted with AES.
*
upload_seconds_delay (Optional. Default: 300): Seconds delay until checking if a new video has been uploaded.
*
debug (Optional. Default: True): Print messages or not.
*
aes_key (Optional. Default: "covert-tube_2021"): Key for AES encryption, used in the "qr_aes" option.
*
generated_video_path (Optional. Default: "output.avi"): Path of video generated with generate_video.py.
*
downloaded_video_path (Optional. Default: "/tmp/test.mp4"): Path where the new video will be downloaded.
*
temp_folder (Optional. Default: "/tmp/"): Path where images of every frame from the video are stored, with the format image_X.png. Installing dependenciesFor all the project:
pip3 install Pillow opencv-python youtube_dl pytesseract pyqrcode pypng pyzbar pycrypto For only the "cleartext" option (OCR): pip3 install Pillow opencv-python youtube_dl pytesseract For only the QR without encryption option: pip3 install Pillow opencv-python youtube_dl pyqrcode pypng pyzbar For only the QR with AES encryption option: pip3 install Pillow opencv-python youtube_dl pyqrcode pypng pyzbar pycrypto Creating a standalone binarypyinstaller --onefile main.py
cp dist/main covert-tube
rm -rf dist build
rm main.spec MotivationLately I have been reading about malware using Youtube for controlling their setting remotely. For example, Casbaneiro abuses YouTube to store its C&C server domains. Each video on the channels used by the threat actor contains a description and at the end of these there is a link to a bogus Facebook or Instagram url containing t[...]___________________________
@hacking_Attack
@Hacking_Video
Kitploit
Kitploit – Maintenance in Progress
Kitploit is temporarily under maintenance. We’ll be back shortly with improvements.
Hacking Articles Tips Tricks Videos Tutorials
KitPloit - PenTest Tools! Covert-Tube - Youtube As Covert-Channel - Control Systems Remotely And Execute Commands By Uploading Videos To Youtube https://1.bp.blogspot.com/-Tkwipt38RVY/YV4Lk3mfEuI/AAAAAAAAwJ4/iSiDlOQuGBUP_O4sdkNdbPG4_-ymCbyhwCNcBGAsYHQ/w640…
he C&C server domain (Welivesecurity blog). A second example is Numando, which abuses it by encrypting the data in the title of the Youtube videos (other Welivesecurity blog).
Knowing this I decided to create a PoC to test the control of remote systems uploading videos to Youtube but, instead of using the title or the description, using the content of the video. It allows to execute any command, but it could be used to change some settings remotely. So this is just a PoC, use it for educational purposes! Download Covert-Tube
___________________________
@hacking_Attack
@Hacking_Video
Knowing this I decided to create a PoC to test the control of remote systems uploading videos to Youtube but, instead of using the title or the description, using the content of the video. It allows to execute any command, but it could be used to change some settings remotely. So this is just a PoC, use it for educational purposes! Download Covert-Tube
___________________________
@hacking_Attack
@Hacking_Video
hacking: security in practice
Hello friends!
I am working on a CTF challenge and I am stuck on a problem.I have a crypted hash I need to decrypt. But I have no hint which crypting is used.It does not look like a base64 or md5.I have 205 characters all upper case A-Z and 0-9 no special characters as sig. and a 8 character key "490d55dc". Is there any website or tool where I can paste my crypted string and it tells me which crypto procedure was possibly used on the string?Thanks for any help!:)
submitted by /u/ValPlusPlusle
[link] [comments]
___________________________
@hacking_Attack
@Hacking_Video
Hello friends!
I am working on a CTF challenge and I am stuck on a problem.I have a crypted hash I need to decrypt. But I have no hint which crypting is used.It does not look like a base64 or md5.I have 205 characters all upper case A-Z and 0-9 no special characters as sig. and a 8 character key "490d55dc". Is there any website or tool where I can paste my crypted string and it tells me which crypto procedure was possibly used on the string?Thanks for any help!:)
submitted by /u/ValPlusPlusle
[link] [comments]
___________________________
@hacking_Attack
@Hacking_Video
reddit
Hello friends!
I am working on a CTF challenge and I am stuck on a problem.I have a crypted hash I need to decrypt. But I have no hint which crypting is used.It...
hacking: security in practice
[Advice] hear me out
So, I do some selling on fb marketplace and I always get people wanting me to ship them products and want to pay me via non-traditional ways. Basically they’re trying to get me to send them my stuff without payment.
I always put in my ad that I will not ship, but that never stops them from trying.
Anyways, yesterday I had a person reach out to me and try to get me to send them my product and once I got the shipping information, the pending payment would go to my acct. they’re expecting and email with the attached receipt.
Now, I’m a nice guy. I don’t do anyone harm on purpose and have a “live and let live” attitude. But, in this particular case, I’m feeling quite vindictive. So, my question is, can someone point me in the direction of figuring out where to get some ransom ware to send them as an attachment to teach them a lesson and help me feel all warm and fuzzy on the inside?
What I would like to do is disguise it as the receipt in an attachment.
Is this even possible? Legal? Or just a cunty thing to do? I don’t mind being petty at this point. What day you folks?
submitted by /u/Battlecat74
[link] [comments]
___________________________
@hacking_Attack
@Hacking_Video
[Advice] hear me out
So, I do some selling on fb marketplace and I always get people wanting me to ship them products and want to pay me via non-traditional ways. Basically they’re trying to get me to send them my stuff without payment.
I always put in my ad that I will not ship, but that never stops them from trying.
Anyways, yesterday I had a person reach out to me and try to get me to send them my product and once I got the shipping information, the pending payment would go to my acct. they’re expecting and email with the attached receipt.
Now, I’m a nice guy. I don’t do anyone harm on purpose and have a “live and let live” attitude. But, in this particular case, I’m feeling quite vindictive. So, my question is, can someone point me in the direction of figuring out where to get some ransom ware to send them as an attachment to teach them a lesson and help me feel all warm and fuzzy on the inside?
What I would like to do is disguise it as the receipt in an attachment.
Is this even possible? Legal? Or just a cunty thing to do? I don’t mind being petty at this point. What day you folks?
submitted by /u/Battlecat74
[link] [comments]
___________________________
@hacking_Attack
@Hacking_Video
reddit
[Advice] hear me out
So, I do some selling on fb marketplace and I always get people wanting me to ship them products and want to pay me via non-traditional ways....
hacking: security in practice
What are the most common hacks businesses deal with?
I’m doing a guidebook in my ethical hacking course, what information should I add along with the question in the title?
submitted by /u/TuneAway
[link] [comments]
___________________________
@hacking_Attack
@Hacking_Video
What are the most common hacks businesses deal with?
I’m doing a guidebook in my ethical hacking course, what information should I add along with the question in the title?
submitted by /u/TuneAway
[link] [comments]
___________________________
@hacking_Attack
@Hacking_Video
reddit
What are the most common hacks businesses deal with?
I’m doing a guidebook in my ethical hacking course, what information should I add along with the question in the title?
hacking: security in practice
welp..
an asshole internet leech hijacked my parents’ facebook page about their farm and is using it for streaming fkn league of legends. I made her change her gmail password but apart from that i dont know what else to do…facebook support is willing to do jack shit so i guess, if someone has any suggestions im open. If you think im not hones, shoot me a dm and i will send you a proof where u can see that their usual content has been a bit…strange lately. Thank you in advance
submitted by /u/Little_Screen4875
[link] [comments]
___________________________
@hacking_Attack
@Hacking_Video
welp..
an asshole internet leech hijacked my parents’ facebook page about their farm and is using it for streaming fkn league of legends. I made her change her gmail password but apart from that i dont know what else to do…facebook support is willing to do jack shit so i guess, if someone has any suggestions im open. If you think im not hones, shoot me a dm and i will send you a proof where u can see that their usual content has been a bit…strange lately. Thank you in advance
submitted by /u/Little_Screen4875
[link] [comments]
___________________________
@hacking_Attack
@Hacking_Video
reddit
welp..
an asshole internet leech hijacked my parents’ facebook page about their farm and is using it for streaming fkn league of legends. I made her...
Hacking Articles Tips Tricks Videos Tutorials
Photo
Kali Linux Tutorials
Kodex : A Privacy And Security Engineering Toolkit: Discover, Understand, Pseudonymize, Anonymize, Encrypt And Securely Share Sensitive And Personal Data: Privacy And Security As Code
Kodex (Community Edition – CE) is an open-source toolkit for privacy and security engineering. It helps you to automate data security and data protection measures in your data engineering workflows. It offers the following functionality:
* Read data items from a variety of sources such as files, databases or message queues.
* Protect these data items using various privacy- & security enhancing transformations, like de-identification, masking, pseudonymization, anonymization or encryption.
* Send the protected items to a variety of destinations.
With Kodex, you can describe your data protection and data security workflows using a simple, declarative configuration language: Just like DevOps tools let you describe infrastructure as code, Kodex is a PrivacyOps & SecurityOps tool that let you describe privacy and security measures as code.
Kodex takes care of the boring and difficult aspects of privacy, such as
* Key management: Kodex manages encryption and pseudonymization keys for you (if you want that).
* Parameter management: Kodex keeps track of how every single data item was processed so you can prove the compliance of your data workflows and create an audit trail.
* Data transformation: Kodex implements modern cryptographic and statistical techniques to protect your data.
Getting started
To download and install Kodex from source, simply run
git clone https://github.com/kiprotect/kodex
cd kiprotect
make
make install
Documentation
CLI
The command line utility is the easiest way to get started analyzing and transforming data with Kodex.
Blueprints
Blueprints are the configuration files that specify how Kodex should load, analyze, transform and write data.
Actions
Actions perform specific operations on input items, e.g. to analyze or transform them. Everything that Kodex does with data is described by an action.
Kodex Community Edition (CE)
To install the community edition of Kodex, simply run
git clone https://github.com/kiprotect/kodex
cd kodex
make
make install
That’s it! You can now process blueprints with the
Kodex Enterprise Edition (EE)
To install the enterprise edition of Kodex (Kodex-EE), first download
kodex-ee install –method docker-compose
cd docker-compose
docker-compose up
That should give you a fully functional test system with the Kodex frontend running on port 4242. The following sections explain in detail how to get Kodex-EE running on your infrastructure in a more permanent way.
Deployment Options
Kodex-EE is deployed as a set of stateless Linux services that interact with external databases and systems, as described above. A full deployment consists of at least one daemon and one API instance. Both of these services are stateless and can be configured using a simple settings file. The API is designed to be used behind a reverse proxy that handles TLS and load balancing (example configurations for servers like nginx or caddy is available).
Requirements
Kodex-EE requires several external services to function:
* A PostgreSQL database (preferrably version 9.6 or newer), which Kodex-EE uses to store meta-data about data streams, actions and parameters. Specific Kodex-EE plugins (e.g. Klaro, Konsens) use their own PostgreSQL schemas for data storage. For stability and perfo[...]
___________________________
@hacking_Attack
@Hacking_Video
Kodex : A Privacy And Security Engineering Toolkit: Discover, Understand, Pseudonymize, Anonymize, Encrypt And Securely Share Sensitive And Personal Data: Privacy And Security As Code
Kodex (Community Edition – CE) is an open-source toolkit for privacy and security engineering. It helps you to automate data security and data protection measures in your data engineering workflows. It offers the following functionality:
* Read data items from a variety of sources such as files, databases or message queues.
* Protect these data items using various privacy- & security enhancing transformations, like de-identification, masking, pseudonymization, anonymization or encryption.
* Send the protected items to a variety of destinations.
With Kodex, you can describe your data protection and data security workflows using a simple, declarative configuration language: Just like DevOps tools let you describe infrastructure as code, Kodex is a PrivacyOps & SecurityOps tool that let you describe privacy and security measures as code.
Kodex takes care of the boring and difficult aspects of privacy, such as
* Key management: Kodex manages encryption and pseudonymization keys for you (if you want that).
* Parameter management: Kodex keeps track of how every single data item was processed so you can prove the compliance of your data workflows and create an audit trail.
* Data transformation: Kodex implements modern cryptographic and statistical techniques to protect your data.
Getting started
To download and install Kodex from source, simply run
git clone https://github.com/kiprotect/kodex
cd kiprotect
make
make install
Documentation
CLI
The command line utility is the easiest way to get started analyzing and transforming data with Kodex.
Blueprints
Blueprints are the configuration files that specify how Kodex should load, analyze, transform and write data.
Actions
Actions perform specific operations on input items, e.g. to analyze or transform them. Everything that Kodex does with data is described by an action.
Kodex Community Edition (CE)
To install the community edition of Kodex, simply run
git clone https://github.com/kiprotect/kodex
cd kodex
make
make install
That’s it! You can now process blueprints with the
kodexCLI command.Kodex Enterprise Edition (EE)
To install the enterprise edition of Kodex (Kodex-EE), first download
kodex-eefrom our web portal. The binary contains everything you need to get started. Notably, it provides an installcommand that (mostly) automates the installation of Kodex-EE for different environments. For example, to create a Docker compose configuration with all Kodex components and required external services, simply runkodex-ee install –method docker-compose
cd docker-compose
docker-compose up
That should give you a fully functional test system with the Kodex frontend running on port 4242. The following sections explain in detail how to get Kodex-EE running on your infrastructure in a more permanent way.
Deployment Options
Kodex-EE is deployed as a set of stateless Linux services that interact with external databases and systems, as described above. A full deployment consists of at least one daemon and one API instance. Both of these services are stateless and can be configured using a simple settings file. The API is designed to be used behind a reverse proxy that handles TLS and load balancing (example configurations for servers like nginx or caddy is available).
Requirements
Kodex-EE requires several external services to function:
* A PostgreSQL database (preferrably version 9.6 or newer), which Kodex-EE uses to store meta-data about data streams, actions and parameters. Specific Kodex-EE plugins (e.g. Klaro, Konsens) use their own PostgreSQL schemas for data storage. For stability and perfo[...]
___________________________
@hacking_Attack
@Hacking_Video
Kali Linux Tutorials
Kodex : A Privacy And Security Engineering Toolkit
Kodex (Community Edition - CE) is an open-source toolkit for privacy and security engineering. It helps you to automate data security.
Hacking Articles Tips Tricks Videos Tutorials
Kali Linux Tutorials Kodex : A Privacy And Security Engineering Toolkit: Discover, Understand, Pseudonymize, Anonymize, Encrypt And Securely Share Sensitive And Personal Data: Privacy And Security As Code Kodex (Community Edition – CE) is an open-source toolkit…
rmance reasons it is advisable to provide separate PostgreSQL instances to these.
* A RabbitMQ server (preferrably version 3.8 or newer), or a Kafka server (preferrably version 2.7 or newer), which Kodex-EE uses to store intermediate data items during processing.
* An authentication service based on SSO, SAML, OpenID-Connect or LDAP, which Kodex-EE can use to authenticate users. The service must provide a mapping of users to organizations and roles within those organizations.
* A Redis database (preferrably version 5.0 or newer), which Kodex-EE uses to e.g. store API and data processing metrics as well as data necessary for rate limiting.
* Optionally, either a persistent directory/volume that is accessible from API and daemon nodes, or a “blob” storage service with an S3-compatible API. Kodex-EE uses these to store file data that is generated for certain background tasks.
The given minimum versions of services are what Kodex-EE is tested and deployed against, it might be possible to run it with older versions as well, though we cannot make any guarantees. The same is true when using newer versions of these services.
PostgreSQL Database(s)
Kodex-EE uses a primary database schema to store meta-data about things like data transformations, sources and destinations. For regular usage, only a small number of table entries are generated, and the primary database schema rarely grows beyond a few GB. Specific Kodex-EE plugins (e.g. Klaro or Konsens) use secondard database schemas to store data results, which grow in proportion to the processed data. Kodex-EE can implement a data retention schedule and will delete old data, nevertheless the required storage will depend on the amount of data processed. Please note that performance may also degrade if very large amounts of data are being processed. All Kodex-EE data schemas are designed to handle billions of rows of data without significant problems though, provided adequate hardware has been provisioned for the databases.
Redis Database(s)
Kodex-EE uses Redis for metrics collection and keeping of internal state during processing (e.g. for anonymization). Metrics are generated using a hierarchical time-spans (hours, days, weeks, months, years) and individual data items expire automatically. Internal state storage requirements will vary depending on the type of data processing and the number of items processed.
Message Queue(s) (RabbitMQ or Kafka)
Kodex-EE stores processed data in a message queue for internal processing and buffering. The amount of data stored in those queues depends on many factors, notably the number of incoming data items, the number of daemon nodes and the read/write capacity of data sources and destinations. Kodex-EE will automatically apply backpressure when writes to the internal queue become impossible.
Daemon & API Requirements
Kodex EE can run on systems with minimal resources like an embedded system or SoC.
Operating System
Kodex-EE is delivered as a statically compiled X64/AMD64 binary that is compatible with all mayor Linux distributions. Running Kodex-EE on Windows, MacOS and OpenBSD should be possible but is currently untested and unsupported.
Memory
We recommend at least 512 MB of available RAM for both the API and the daemon. Actual memory usage can vary depending on the workloads but Kodex EE will try to automatically limit data processing in order to not exhaust all available memory.
CPU
Both the daemon and the API process data concurrently in parallel threads, hence they can effectively utilize multiple CPU cores / processing threads.
Storage
Apart from task-specific storage mentioned above, neither the daemon nor the API require persistent on-disk data storage.
Networking
Kodex-EE needs to have connectivity to data sources and destinations from which it should read or to which it should write data. Processing speed will be limited by available network bandwidth.
Operations
Th[...]
___________________________
@hacking_Attack
@Hacking_Video
* A RabbitMQ server (preferrably version 3.8 or newer), or a Kafka server (preferrably version 2.7 or newer), which Kodex-EE uses to store intermediate data items during processing.
* An authentication service based on SSO, SAML, OpenID-Connect or LDAP, which Kodex-EE can use to authenticate users. The service must provide a mapping of users to organizations and roles within those organizations.
* A Redis database (preferrably version 5.0 or newer), which Kodex-EE uses to e.g. store API and data processing metrics as well as data necessary for rate limiting.
* Optionally, either a persistent directory/volume that is accessible from API and daemon nodes, or a “blob” storage service with an S3-compatible API. Kodex-EE uses these to store file data that is generated for certain background tasks.
The given minimum versions of services are what Kodex-EE is tested and deployed against, it might be possible to run it with older versions as well, though we cannot make any guarantees. The same is true when using newer versions of these services.
PostgreSQL Database(s)
Kodex-EE uses a primary database schema to store meta-data about things like data transformations, sources and destinations. For regular usage, only a small number of table entries are generated, and the primary database schema rarely grows beyond a few GB. Specific Kodex-EE plugins (e.g. Klaro or Konsens) use secondard database schemas to store data results, which grow in proportion to the processed data. Kodex-EE can implement a data retention schedule and will delete old data, nevertheless the required storage will depend on the amount of data processed. Please note that performance may also degrade if very large amounts of data are being processed. All Kodex-EE data schemas are designed to handle billions of rows of data without significant problems though, provided adequate hardware has been provisioned for the databases.
Redis Database(s)
Kodex-EE uses Redis for metrics collection and keeping of internal state during processing (e.g. for anonymization). Metrics are generated using a hierarchical time-spans (hours, days, weeks, months, years) and individual data items expire automatically. Internal state storage requirements will vary depending on the type of data processing and the number of items processed.
Message Queue(s) (RabbitMQ or Kafka)
Kodex-EE stores processed data in a message queue for internal processing and buffering. The amount of data stored in those queues depends on many factors, notably the number of incoming data items, the number of daemon nodes and the read/write capacity of data sources and destinations. Kodex-EE will automatically apply backpressure when writes to the internal queue become impossible.
Daemon & API Requirements
Kodex EE can run on systems with minimal resources like an embedded system or SoC.
Operating System
Kodex-EE is delivered as a statically compiled X64/AMD64 binary that is compatible with all mayor Linux distributions. Running Kodex-EE on Windows, MacOS and OpenBSD should be possible but is currently untested and unsupported.
Memory
We recommend at least 512 MB of available RAM for both the API and the daemon. Actual memory usage can vary depending on the workloads but Kodex EE will try to automatically limit data processing in order to not exhaust all available memory.
CPU
Both the daemon and the API process data concurrently in parallel threads, hence they can effectively utilize multiple CPU cores / processing threads.
Storage
Apart from task-specific storage mentioned above, neither the daemon nor the API require persistent on-disk data storage.
Networking
Kodex-EE needs to have connectivity to data sources and destinations from which it should read or to which it should write data. Processing speed will be limited by available network bandwidth.
Operations
Th[...]
___________________________
@hacking_Attack
@Hacking_Video
Hacking Articles Tips Tricks Videos Tutorials
rmance reasons it is advisable to provide separate PostgreSQL instances to these. * A RabbitMQ server (preferrably version 3.8 or newer), or a Kafka server (preferrably version 2.7 or newer), which Kodex-EE uses to store intermediate data items during processing.…
e following sections describe operations-related aspects of Kodex-EE.
Metrics
Kodex-EE collects various metrics that are exposed via the REST API and supports instrumentation for Prometheus.
Logging
Kodex-EE has an internal, level-based logging system that is compatible with syslog.
Command Line Interface (CLI)
The Kodex command line interface (CLI) is the easiest way to get started with privacy and security engineering. It enables you to run a wide range of privacy- & security-enhancing transformations as well as analyses on your structured data.
Installing Kodex CLI
First, you need to download or build the Kodex CLI tool. You can download pre-built binaries for various platforms on our website. Alternatively, you can build the tool from source by following the instructions on our Github page.
Getting started
By default, we control the CLI tool using so-called blueprints. A blueprint is a config file (or a collection of them) that describes how Kodex should read, analyze, transform and write structured data.
To run a blueprint, you simply execute
kodex blueprints download
This will download our public blueprints repository and store it in a local directory (by default
kodex run pseudonymization/examples/data-types/pseudonymize
This will load the configuration from the blueprint file (
The items in the input.json file that we want to pseudonymize.
{
“name”: “test”,
“date”: “2020-06-04”,
“ip”: “42.34.122.112/32”,
“count”: 4354
}
{
“name”: “another test”,
“date”: “2019-07-02”,
“ip”: “42.34.122.114/32”,
“count”: 214
}
The example blueprint that we picked will read data items from an
{
“_kip”: “278ba5f7db26ca661b4e64b1eb6abb3d4e7d1aa15e55155b3a1f7626424f679c”,
“count”: 7643,
“date”: “2021-09-11”,
“ip”: “150.39.196.226/32”,
“name”: “3YdcJQ==”
}
{
“_kip”: “278ba5f7db26ca661b4e64b1eb6abb3d4e7d1aa15e55155b3a1f7626424f679c”,
“count”: 46,
“date”: “2015-07-19”,
“ip”: “150.39.196.8/32”,
“name”: “gKGhSKShnHg/vny6”
}
As you can see, Kodex pseudonymized every attribute in every data item and also added a new attribute,
Depseudonymizing data
At some point you might actually want to depseudonymize your data again. Kodex makes this easy by providing an
___________________________
@hacking_Attack
@Hacking_Video
Metrics
Kodex-EE collects various metrics that are exposed via the REST API and supports instrumentation for Prometheus.
Logging
Kodex-EE has an internal, level-based logging system that is compatible with syslog.
Command Line Interface (CLI)
The Kodex command line interface (CLI) is the easiest way to get started with privacy and security engineering. It enables you to run a wide range of privacy- & security-enhancing transformations as well as analyses on your structured data.
Installing Kodex CLI
First, you need to download or build the Kodex CLI tool. You can download pre-built binaries for various platforms on our website. Alternatively, you can build the tool from source by following the instructions on our Github page.
Getting started
By default, we control the CLI tool using so-called blueprints. A blueprint is a config file (or a collection of them) that describes how Kodex should read, analyze, transform and write structured data.
To run a blueprint, you simply execute
kodex run [blueprint name]. Kodex comes with a free, public repository of example blueprints that help us to get started. We can download and install them via the command line as well:kodex blueprints download
This will download our public blueprints repository and store it in a local directory (by default
~/.kodex/blueprints). You can then run any blueprint by simply specifying its path relative to the blueprints directory. So let’s run a simple example that shows how Kodex can pseudonymize different data types:kodex run pseudonymization/examples/data-types/pseudonymize
This will load the configuration from the blueprint file (
pseudonymize.yml). This file specifies from where data should be read (a JSON file in this case), how the data should be transformed (using a pseudonymization in this case) and where the resulting output data should be sent to (a JSON file again).The items in the input.json file that we want to pseudonymize.
{
“name”: “test”,
“date”: “2020-06-04”,
“ip”: “42.34.122.112/32”,
“count”: 4354
}
{
“name”: “another test”,
“date”: “2019-07-02”,
“ip”: “42.34.122.114/32”,
“count”: 214
}
The example blueprint that we picked will read data items from an
input.jsonfile located in the same directory as the blueprint, pseudonymize all attributes of each item using different applicable pseudonymization methods, and write the pseudonymized data to a JSON file (pseudonymized.json) in the current directory. Here’s what the output looks like:{
“_kip”: “278ba5f7db26ca661b4e64b1eb6abb3d4e7d1aa15e55155b3a1f7626424f679c”,
“count”: 7643,
“date”: “2021-09-11”,
“ip”: “150.39.196.226/32”,
“name”: “3YdcJQ==”
}
{
“_kip”: “278ba5f7db26ca661b4e64b1eb6abb3d4e7d1aa15e55155b3a1f7626424f679c”,
“count”: 46,
“date”: “2015-07-19”,
“ip”: “150.39.196.8/32”,
“name”: “gKGhSKShnHg/vny6”
}
As you can see, Kodex pseudonymized every attribute in every data item and also added a new attribute,
_kip, to the items. The value of that attribute refers to a parameter set that contains the cryptographic keys that were used to transform the data. The actual keys are stored in a so-called parameter store. If you don’t want that, you can also manage keys yourself as well: The pseudonymize-with-keyblueprint in the same directory does that by first asking you to enter a key and then using that key to derive further encryption keys for the individual pseudonymization operations. Key & parameter management is a complex topic in itself, for now just rest assured that Kodex takes care of the messy details for you.Depseudonymizing data
At some point you might actually want to depseudonymize your data again. Kodex makes this easy by providing an
undoaction that can be applied to reversible transformations like the cryptog[...]___________________________
@hacking_Attack
@Hacking_Video
Hacking Articles Tips Tricks Videos Tutorials
e following sections describe operations-related aspects of Kodex-EE. Metrics Kodex-EE collects various metrics that are exposed via the REST API and supports instrumentation for Prometheus. Logging Kodex-EE has an internal, level-based logging system…
raphic pseudonymization above. So, to depseudonymize the data above, we can simply run a blueprint that contains such an
kodex run pseudonymization/examples/data-types/depseudonymize
which will print the depseudonymized data (which should exactly match the input data). If you provided a pseudonymization key yourself by using the
Actions
Actions are how Kodex transforms or analyzes data. Actions can be called individually or as a sequence, in which each action will receive the result of the previous action. The following sections describe the different action types that Kodex currently supports.
Generic Validation & Transformation
Often data is complex & hierarchical, making it difficult to apply simple transformations to it. For this case Kodex support generic form-based validation and transformation of data. This enables you to easily parse, validate and transform complex, hierarchical data like JSON documents. You can read more about this type of transformation in the form action documenation.
Pseudonymization
Pseudonymization produces data that is no longer directly attributable to a specific individual. Using pseudonymous data lowers the risk of data processing for individuals and reduces the impact of data loss or data theft. It can be applied to direct or indirect identifiers as well as to a wide range of structured data types like numbers, dates, names or IP addresses. Some pseudonymization methods are based on reversible encryption, which makes it possible to de-pseudonymize the data again given knowledge of the key. Other methods like hashing are non-reversible.
Encryption
Encryption produces data that is statistically indistinguishable from random noise and that can be decrypted only with knowledge of a secret encryption key. Kodex implements standard symmetric and asymmetric encryption techniques.
Please note that we consider encryption methods which produce the same ciphertext when given identical input data and encryption key(s) as pseudonymization methods, as the resulting data does (intentionally) not conform to modern security standards for encryption methods. Therfore, format-preserving encryption methods that operate without random initialization vectors (IV) are also considered pseudonymization methods under this approach.
Anonymization
Anonymization actions use statistical techniques to produce data from which it it should not be possible to identify any specific individuals or infer any non-trivial information about these individual. Kodex relies on randomization and aggregation as anonymity mechanisms and produces anonymous data that conforms to modern anonymity standards like “differential privacy”.
Discovery
Discovery actions enable you to detect different types of personal or sensitive information in your structured and unstructured data items.
Identification & Identity Management
Identification & identity management actions enable you to identify individuals based on various direct or indirect identifiers, and associate a permanent pseudonymous ID to every individual. This allows you to e.g. attribute data items from different sources to a single individual and use the pseudonymous ID of that individual to e.g. produce anonymous or pseudonymous data or verify the consent of that individual for a given data processing purpose.
Consent Management
Consent management actions enable you to verify that a given individual has given consent for a specific data processing purpose. Together with the identification & identity management actions they enable you to build compliant data processing workflows and e.g. surpress data from individuals that have not given or withdrawn their consent for a specific processing purpose.
Audit Logging
Audit l[...]
___________________________
@hacking_Attack
@Hacking_Video
undoaction:kodex run pseudonymization/examples/data-types/depseudonymize
which will print the depseudonymized data (which should exactly match the input data). If you provided a pseudonymization key yourself by using the
pseudonymize-with-key, you can run depseudonymize-with-keyblueprint instead, which will ask you to enter the pseudonymization key. Isn’t that easy?Actions
Actions are how Kodex transforms or analyzes data. Actions can be called individually or as a sequence, in which each action will receive the result of the previous action. The following sections describe the different action types that Kodex currently supports.
Generic Validation & Transformation
Often data is complex & hierarchical, making it difficult to apply simple transformations to it. For this case Kodex support generic form-based validation and transformation of data. This enables you to easily parse, validate and transform complex, hierarchical data like JSON documents. You can read more about this type of transformation in the form action documenation.
Pseudonymization
Pseudonymization produces data that is no longer directly attributable to a specific individual. Using pseudonymous data lowers the risk of data processing for individuals and reduces the impact of data loss or data theft. It can be applied to direct or indirect identifiers as well as to a wide range of structured data types like numbers, dates, names or IP addresses. Some pseudonymization methods are based on reversible encryption, which makes it possible to de-pseudonymize the data again given knowledge of the key. Other methods like hashing are non-reversible.
Encryption
Encryption produces data that is statistically indistinguishable from random noise and that can be decrypted only with knowledge of a secret encryption key. Kodex implements standard symmetric and asymmetric encryption techniques.
Please note that we consider encryption methods which produce the same ciphertext when given identical input data and encryption key(s) as pseudonymization methods, as the resulting data does (intentionally) not conform to modern security standards for encryption methods. Therfore, format-preserving encryption methods that operate without random initialization vectors (IV) are also considered pseudonymization methods under this approach.
Anonymization
Anonymization actions use statistical techniques to produce data from which it it should not be possible to identify any specific individuals or infer any non-trivial information about these individual. Kodex relies on randomization and aggregation as anonymity mechanisms and produces anonymous data that conforms to modern anonymity standards like “differential privacy”.
Discovery
Discovery actions enable you to detect different types of personal or sensitive information in your structured and unstructured data items.
Identification & Identity Management
Identification & identity management actions enable you to identify individuals based on various direct or indirect identifiers, and associate a permanent pseudonymous ID to every individual. This allows you to e.g. attribute data items from different sources to a single individual and use the pseudonymous ID of that individual to e.g. produce anonymous or pseudonymous data or verify the consent of that individual for a given data processing purpose.
Consent Management
Consent management actions enable you to verify that a given individual has given consent for a specific data processing purpose. Together with the identification & identity management actions they enable you to build compliant data processing workflows and e.g. surpress data from individuals that have not given or withdrawn their consent for a specific processing purpose.
Audit Logging
Audit l[...]
___________________________
@hacking_Attack
@Hacking_Video
Hacking Articles Tips Tricks Videos Tutorials
raphic pseudonymization above. So, to depseudonymize the data above, we can simply run a blueprint that contains such an undoaction: kodex run pseudonymization/examples/data-types/depseudonymize which will print the depseudonymized data (which should exactly…
ogging actions enable you to trace the flow of data from individuals through your entire data processing infrastructure. They produce pseudonymous, searchable information about how data belonging to a specific individual was processed and where it was sent to, allowing you to e.g. retrieve, amend or delete this data.
Form Action
The
Getting Started
A form consists of a number of fields. Each field can define one or more validators, which can validate and transform the content of the field. If the form validation is successful, the action will return the validated fields. If not, an error will be thrown. Here’s an example form action specification:
type: form
config:
fields:
– name: date
validators:
– type: IsString
– type: IsTime
config:
format: rfc3339
– name: count
validators:
– type: IsInteger
config:
hasMin: true
min: 0
hasMax: true
max: 1000
This action will ensure each item has a
* CanBeAnything: Just ensures the field is present, but does not perform any other validation on it.
* IsBoolean: Ensures the field contains a boolean value, i.e.
* IsBytes: Ensures the field contains a byte array with a given
* IsFloat: Ensures the field contains a float value. Optionally, if
* IsHex: Ensures the field contains a hex value. If
* IsIn: Ensures the field contains one of the specified
* IsInteger: Ensure the filed contains an integer value. Optionally, if
* IsList: Ensures the field contains a list of values. Optionally, a list of
* IsNotIn: Like IsIn, but ensures the value of the field is not in the given
* IsOptional: Will skip the other validators if the field is undefined, making it optional.
* IsString: Ensures the field contains a string value. Optionally,
* IsStringList: Ensures the field contains a list of strings. Optionally, a list of
* IsStringMap: Ensures the field contains a string-based hash map / dictionary. Optionally, a
* IsTime: Ensures the field contains a datetime/time object in the given
* IsUUID: Ensures the field contains a UUID value.
* MatchesRegex: Ensures the field matches the given regular expression. Assumes that the field contains a string value, which should be validated before using the
* Or: Ensures the field contains one of several possible values as specified by a list of
___________________________
@hacking_Attack
@Hacking_Video
Form Action
The
formaction implements our open-source form validation & transformation library, enabling you to parse, validate and transform complex, hierarchical data.Getting Started
A form consists of a number of fields. Each field can define one or more validators, which can validate and transform the content of the field. If the form validation is successful, the action will return the validated fields. If not, an error will be thrown. Here’s an example form action specification:
type: form
config:
fields:
– name: date
validators:
– type: IsString
– type: IsTime
config:
format: rfc3339
– name: count
validators:
– type: IsInteger
config:
hasMin: true
min: 0
hasMax: true
max: 1000
This action will ensure each item has a
datefield that contains an RFC-3339 formatted data string and a countfield that contains an integer value between 0 and 1000. There are many other validator types available, here’s the full list:* CanBeAnything: Just ensures the field is present, but does not perform any other validation on it.
* IsBoolean: Ensures the field contains a boolean value, i.e.
trueor false.* IsBytes: Ensures the field contains a byte array with a given
encoding, e.g. hex, base64or base64-urlencoded data.* IsFloat: Ensures the field contains a float value. Optionally, if
hasMinor hasMaxis true, the minor maxconfig options define the range of the float value.* IsHex: Ensures the field contains a hex value. If
convertis true, the value will be converted to a byte array.* IsIn: Ensures the field contains one of the specified
choices.* IsInteger: Ensure the filed contains an integer value. Optionally, if
hasMinor hasMaxis true, the minor maxconfig options define the range of the integer value.* IsList: Ensures the field contains a list of values. Optionally, a list of
validatorswill be applied to each list element.* IsNotIn: Like IsIn, but ensures the value of the field is not in the given
choices.* IsOptional: Will skip the other validators if the field is undefined, making it optional.
* IsString: Ensures the field contains a string value. Optionally,
minLengthand maxLengthspecify the minimum and maximum length of the string.* IsStringList: Ensures the field contains a list of strings. Optionally, a list of
validatorswill be applied to each list element.* IsStringMap: Ensures the field contains a string-based hash map / dictionary. Optionally, a
formcan be specified that will be used to validate the value.* IsTime: Ensures the field contains a datetime/time object in the given
format, either rfc3339, rfc3339-date, unix, unix-nanoor unix-milli. If rawis true the string will not be converted to a Timeobject. If toUTCis true, the resulting Timeobject will be converted to UTC.* IsUUID: Ensures the field contains a UUID value.
* MatchesRegex: Ensures the field matches the given regular expression. Assumes that the field contains a string value, which should be validated before using the
IsStringvalidator.* Or: Ensures the field contains one of several possible values as specified by a list of
options, which in turn contain a [...]___________________________
@hacking_Attack
@Hacking_Video
Hacking Articles Tips Tricks Videos Tutorials
ogging actions enable you to trace the flow of data from individuals through your entire data processing infrastructure. They produce pseudonymous, searchable information about how data belonging to a specific individual was processed and where it was sent…
list of validators. Will stop at the first option that successfully validates the value.
* Switch: Validates the field against a list of validators defined in a
Actions Within Forms
In addition, the form mechanism supports all other actions as well through the
type: IsAction
config:
type: pseudonymize
config:
method: merengue
This performs a pseudonymization of the value using the
Blueprints
Blueprints are configurations that specify how Kodex should read, analyze transform and write data.
Privacy & security engineering is complex: In a typical scenario, we might want to read data from a variety of sources (e.g. a database, file or API) and process it for different purposes (e.g. analytics or anomaly detection), which often require different trasformations. We also might want to send the data to a variety of destinations (e.g. a database or message queue). Finally, we might want to do all this continously as data is produced.
To make this as easy as possible, we implemented a declarative and expressive config language based on YAML files. Other systems like Kubernetes or Ansible have shown that collections of YAML files can be powerful tools for configuration management. Also, files are easy to work with and can be versioned just like any other configuration file.
Blueprint basics
The basic structure of a blueprint. This example would read data from a SQL database, pseudonymize it and forward the pseudonymized data to a HTTP API.
actions: # different actions we want to apply to our data items
* name: pseudonymize-name
type: pseudonymize
…
sources: # sources we want to load data items from
* name: production-db
type: sql
…
destinations: # destinations we want to send data items to
* name: audit-api
type: http
…
streams: # streams of data items that we want to process
* name: default description: | Pseudonymize incoming data before sending it to the audit API. sources: # sources of data items for this stream
* production-db
configs: # configurations for this stream
* name: default actions: # actions in this configuration
* pseudonymize-name
destinations: # destinations that these items should be sent to
* name: audit-api
A blueprint essentially contains at least four different sections: Actions, sources, destinations and streams. Each stream refers to one or more sources and contains one or more stream configs. Each stream config refers to one or more actions as well as one or more destinations. Together these primitives tell Kodex how data should flow through it and how it should be analyzed and transformed.
Running blueprints
You can run a given blueprint using the
kodex run [blueprint name]
If you omit the
kodex run pseudonymization/examples/data-types/pseudonymize
Kodex will try to find a blueprint named
kodex run my-blueprint –[...]
___________________________
@hacking_Attack
@Hacking_Video
* Switch: Validates the field against a list of validators defined in a
casesstring map, where the chosen case depends on the value of another field whose name is given by key.Actions Within Forms
In addition, the form mechanism supports all other actions as well through the
IsActionvalidator, which takes an action specification as configuration. Example validator:type: IsAction
config:
type: pseudonymize
config:
method: merengue
This performs a pseudonymization of the value using the
merenguemethod. Parameters of embedded actions are managed automatically by the form action.Blueprints
Blueprints are configurations that specify how Kodex should read, analyze transform and write data.
Privacy & security engineering is complex: In a typical scenario, we might want to read data from a variety of sources (e.g. a database, file or API) and process it for different purposes (e.g. analytics or anomaly detection), which often require different trasformations. We also might want to send the data to a variety of destinations (e.g. a database or message queue). Finally, we might want to do all this continously as data is produced.
To make this as easy as possible, we implemented a declarative and expressive config language based on YAML files. Other systems like Kubernetes or Ansible have shown that collections of YAML files can be powerful tools for configuration management. Also, files are easy to work with and can be versioned just like any other configuration file.
Blueprint basics
The basic structure of a blueprint. This example would read data from a SQL database, pseudonymize it and forward the pseudonymized data to a HTTP API.
actions: # different actions we want to apply to our data items
* name: pseudonymize-name
type: pseudonymize
…
sources: # sources we want to load data items from
* name: production-db
type: sql
…
destinations: # destinations we want to send data items to
* name: audit-api
type: http
…
streams: # streams of data items that we want to process
* name: default description: | Pseudonymize incoming data before sending it to the audit API. sources: # sources of data items for this stream
* production-db
configs: # configurations for this stream
* name: default actions: # actions in this configuration
* pseudonymize-name
destinations: # destinations that these items should be sent to
* name: audit-api
A blueprint essentially contains at least four different sections: Actions, sources, destinations and streams. Each stream refers to one or more sources and contains one or more stream configs. Each stream config refers to one or more actions as well as one or more destinations. Together these primitives tell Kodex how data should flow through it and how it should be analyzed and transformed.
Running blueprints
You can run a given blueprint using the
runcommand:kodex run [blueprint name]
If you omit the
[blueprint name], Kodex will look for a file named .blueprint.ymlin the current directory to run. Otherwise, Kodex will first check if you entered a file location, and if so loads the blueprint from there. Otherwise, Kodex will go through all of its blueprint directories (by default ~/.kodex/blueprints) and try to find the blueprint you specified. For example, if we runkodex run pseudonymization/examples/data-types/pseudonymize
Kodex will try to find a blueprint named
pseudonymize.ymlin a subfolder pseudonymization/examples/data-typeswithin any of the given blueprint paths. Sometimes you might have different versions of a blueprint installed. By default, Kodex will load the latest version it can find. If you don’t want that, you can specify a version using the --versionflag:kodex run my-blueprint –[...]
___________________________
@hacking_Attack
@Hacking_Video
Hacking Articles Tips Tricks Videos Tutorials
list of validators. Will stop at the first option that successfully validates the value. * Switch: Validates the field against a list of validators defined in a casesstring map, where the chosen case depends on the value of another field whose name is given…
version 0.4.1
Version numbers follow the semantic version specification (2.0).
Blueprint repositories
Kodex lets you download and install blueprints from so-called repositories. For example, to get our official blueprints repository, simply run
This will download the latest snapshot of the master branch of our Blueprints repository and install it into a local directory. If you want to download blueprints from a different URL simple provide it:
kodex blueprints download https://my.blueprints/repo.zip
Kodex will look for a directory with a
package: [your-package-name] # e.g. my-blueprints
version: [your version] # e.g. 1.5.2 Download
___________________________
@hacking_Attack
@Hacking_Video
Version numbers follow the semantic version specification (2.0).
Blueprint repositories
Kodex lets you download and install blueprints from so-called repositories. For example, to get our official blueprints repository, simply run
This will download the latest snapshot of the master branch of our Blueprints repository and install it into a local directory. If you want to download blueprints from a different URL simple provide it:
kodex blueprints download https://my.blueprints/repo.zip
Kodex will look for a directory with a
.blueprints.ymlfile in the ZIP archive and extract that directory to the main blueprints path. You can also create own local blueprint repositories of course, just make sure to put them in a subfolder of a blueprints path and create a .blueprints.ymlfile with the following content:package: [your-package-name] # e.g. my-blueprints
version: [your version] # e.g. 1.5.2 Download
___________________________
@hacking_Attack
@Hacking_Video
Hacking Articles Tips Tricks Videos Tutorials
Photo
Hacking on Medium
How I accidently Exploited My library Computers…
“Now, this is a story all about how
My life got flipped-turned upside down
And I liked to take a minute
Just sit right there
I’ll tell you…
Continue reading on Medium »
___________________________
@hacking_Attack
@Hacking_Video
How I accidently Exploited My library Computers…
“Now, this is a story all about how
My life got flipped-turned upside down
And I liked to take a minute
Just sit right there
I’ll tell you…
Continue reading on Medium »
___________________________
@hacking_Attack
@Hacking_Video
Medium
How I accidently Exploited My library Computers…
“Now, this is a story all about how My life got flipped-turned upside down And I liked to take a minute Just sit right there I’ll tell you…