View open ports without netstat or other tool:

# Get all open ports in hex format
declare -a open_ports=($(cat /proc/net/tcp | grep -v "local_address" | awk '{ print $2 }' | cut -d':' -f2))
# Show all open ports and decode hex to dec
for port in ${open_ports[*]}; do echo $((0x${port})); done

#linux #ports #netstat #tcp #open_ports #sysadmin

https://www.aquasec.com/wiki/display/containers/Basic+Docker+Operations

#docker #container #wiki

#fun #programmer_jokes

How to truncate a log file in Linux:

> logfile

or

cat /dev/null > logfile

If you want to be more eloquent, will empty logfile (actually they will truncate it to zero size). If you want to know how long it "takes", you may use

dd if=/dev/null of=logfile

(which is the same as dd if=/dev/null > logfile, by the way)

You can also use:

truncate logfile --size 0

to be perfectly explicit or, if you don't want to

rm logfile

(applications usually do recreate a logfile if it doesn't exist already).

However, since logfiles are usually useful, you might want to compress and save a copy. While you could do that with your own script, it is a good idea to at least try using an existing working solution, in this case logrotate, which can do exactly that and is reasonably configurable.

#linux #sysadmin #truncate #dd #dev_null #logfile

How to check whether your site is served in gzip or not by using cURL linux command:

curl -H "Accept-Encoding: gzip" -I https://www.google.com

If you find the bloew line in the returned response, your site supports gzip:

content-encoding: gzip

NOTE: gzip can speed up your site speed tremendously, sometimes by 90% compression! It also affects your google rank.

#linux #curl #gzip #seo

-> VERY IMPORTANT <-

How to solve a technical problem in any scale?

Keep Calm:
Keep calm and focus. The first and the most important factor of solving a problem is to get a grip on yourself and get confident and focused. They are excited, angry or maybe out of control. If you get angry too, or excited, solving the problem will get more time than you would think of. Believe you me!

Truthify:
Is the problem explained is really the problem that has happened?
Most of the time problems are explained in a non-technical, wrong way! You are responsible to question him/her to see what actually has happened? You are the detective of the crime scene.

Isolation:
Is there any possibilty to isolate the problem from the sorroundings. Finding the real problem when different metrics affect the result will guide to a wrong path. For example if it is said that emails are not sent, try to just send email from mail server to see if everything is working, put aside your web application entirely to detect the root cause of the problem. Or when it is said that no one can upload files, check your server free disk space, inode, etc.

Cross Check:
To solve a problem try to test the result from different sources. If for example your payments are stored in 3 places (for BI or even the architecture needs in 2 places in different forms) check them all over again from bottom up and vice versa. Or if you have the app in different platforms, check them from web, iOS, android to see whether problem is platform-dependent or not.

#problem_solving #truthify #keep_calm #isolation #cross_check

#ebbok #epub #book #tensor_flow #deep_learning

simple introduction to Cement framework and its usage. Cement framework is mostly used for creating a command line application using python.

In version 2.6 of Cement you can initiate an app using with:

from cement.core.foundation import CementApp

 with CementApp('myapp') as app:
     app.run()

It hides complexieties in your application initiation. That is the above code would be something like below without using with:

from cement.core.foundation import CementApp

 app = CementApp('myapp')
 app.setup()
 app.run()
 app.close()

As you can see with procedure is more clear and straight forward with less code. I know it's silly to have an app like above, but that's just an introduction to the world of Cement.

To add logger to your framework you need to set log level in log.logging as below:

from cement.utils.misc import init_defaults
 from cement.core.foundation import CementApp

 defaults = init_defaults('myapp', 'log.logging')
 defaults['log.logging']['level'] = 'DEBUG'
 defaults['log.logging']['file'] = 'cementy.log'
 defaults['log.logging']['to_console'] = True

 with CementApp('myapp', config_defaults=defaults) as app:
     app.run()
     app.log.debug('This is debug')

init_defaults is used to setup logging. level sets the log level to DEBUG. file would write log data into cementy.log file.
By setting to_console param you can also write the data written to file into console too. So if you run your python application, a file would be created for logging and data will be printed out.

#python #cement #framework #logging #log #level #foundation

#ebook #epub #book #python #effective_python

To get on going processes in mysql client and see which queries are taking longer use:

SHOW PROCESSLIST;

It will show you a table with list of all connection from different hosts (if applicable) and their PID number. You can use this number to kill a process that consumes your server CPU, Memory, etc.:

KILL <pid>;

#mysql #client #kill #processlist #sysadmin #dba #linux

nmap (Network Mapper) is a security scanner, originally written by Gordon Lyon.

Nmap features include:
Host discovery – Identifying hosts on a network. For example, listing the hosts that respond to TCP and/or ICMP requests or have a particular port open.

Port scanning – Enumerating the open ports on target hosts.

Version detection – Interrogating network services on remote devices to determine application name and version number.[7]

OS detection – Determining the operating system and hardware characteristics of network devices.

Scriptable interaction with the target – using Nmap Scripting Engine[8] (NSE) and Lua programming language.

If you want to check whether a port on remote host is open:

sudo nmap -v -p 80 google.com

In response it shows that the port is open/closed/filtered:

Not shown: 987 filtered ports
 20/tcp   closed ftp-data
 80/tcp   open   http

In case you want to scan all port using nmap:

sudo nmap -v your_target_victim.com

It lists all the gathered information about open ports.

#nmap #security #port #open #open_port #port_scanner

https://github.com/darklow/django-suit/

#django #python #django_suit

If you are a Python programmer and need a reference to almost every useful python package, take look at the below repo in github:

https://github.com/vinta/awesome-python

#python #reference #github #awesome_python

urlparse library is used in python in order to well, parse a URL.

Consider the example below:

from urlparse import urlparse
 google_url = urlparse('https://www.google.com/profile?active=true#comment')

google_url is now of type ParseResult class, which includes all data we need:

The protocol used in URL is achievable by:

print url.scheme
 'https'

Absolute url of the google url is retrieved by netloc:

print url.netloc
 'www.google.com'

The path section of the URL which is after the domain section is:

print url.path
 '/profile'

Query section of the URL:

print url.query
 'active=true'

The fragment part of url is stored in fragment attribute:

print url.fragment
 'comment'

#python #urlparse #url

watch linux command is used to run a command at regular intervals.

The command below is the simplest form of watch:

watch YOUR_COMMAND

For instance:

watch df -h

The command above runs df -h (check disk space) every 2 seconds by default.

In order to change the interval:

watch -n 5 df -h

-n or --interval specify update interval in second. The command will not allow quicker than 0.1 second interval.

In case you want to see the differences in your output command use -d or --differences. It wil highlight
when part of your command output changes. For example in our command if disk space usage changes we will see
the new result highlighted.


SIDE NOTE: -h in df command will show a human readable format of disk space in mega byte.


#linux #sysadmin #watch

minimum interval of 0.1 is hardcoded in the source code of watch command in the link below line 171:

http://procps.cvs.sourceforge.net/viewvc/procps/procps/watch.c?view=markup

Thousand separator using format in python:

your_number = 35200000
 print '{:,}'.format(your_number)

#python #format #thousand_separator

A comprehensive management system for Apache Kafka®

https://www.confluent.io/product/control-center/

#kafka #confluent #monitoring

If you want to run a script, ALWAYS log script output into a file or you will be bitten in the ass and would not have any log data for future reference.

In a regular whay when you try to run a python script you would use:

python my_script.py

Anything that will be printed inside of the script will be printed out into the stdout, so you use the below code to put the script output (stdout) into a file:

python my_script.py >> my_script.log

The above command will put the output into a persisted file that can be referenced in the future.

NOTE: The above scenario is for cases when you don't use a log handler in your script, or when
you are in a hurry and just want to put output in a file. Logging solution is definitely a good s
olution.

Finally if you want to run the script in background use:

python my_script.py >> my_script.log 2>&1

2>&1: 1 is for stdin and 2 is for stderr( if exist code of non-success happens). This command s
ays that send stderr messages into stdout.

#linux #python #script #log

You can open new tabs in vim using tabe command:

:tabe

At the top of the vim you would see [No Name] which refers to the current file
name. Now in order to navigate between tabs you can use the below command when your vim is in command mode:

gt

To go to the previous tab use:

gT

#vim #tab #multi_window #gt #tabe #tips

https://stackoverflow.com/questions/32461229/why-use-redux-over-facebook-flux

#react #reactjs #flux #redux