Nice read about systemctl

https://www.digitalocean.com/community/tutorials/how-to-use-systemctl-to-manage-systemd-services-and-units

#linux

A draft of the article about DS Bowl 2018 on Kaggle.
This time this was a lottery.

Good that I did not really spend much time, but this time I learned a lot about watershed and some other instance segmentation methods!

An article is accompanied by a dockerized PyTorch code release on GitHub:
- https://spark-in.me/post/playing-with-dwt-and-ds-bowl-2018
- https://github.com/snakers4/ds_bowl_2018

This is a beta, you are welcome to comment and respond.
Kudos!

#data_science
#deep_learning
#instance_se

Also what is interesting, despite the fact that geektimes blocked my SOCKS proxy post and the fact that marketing based web-sites stole it (in Russian), I received the following feedback:
- 3 people thanked me in the ODS channel
- 3 people thanked me via email
- 2 people thanked me in geektimes PM

Also this is also interesting - my referral link was hit 165 times and ~50 people registered =)
- http://prntscr.com/j69w85

So if you missed the fun
- Post https://spark-in.me/post/vds-socks5-proxy-server
- Referral link https://m.do.co/c/6f8e77dddc23
- Note that the final config is in the comments and here (thanks to https://t.me/bykvaadm and its admin)

sudo apt update && apt upgrade

wget https://launchpad.net/ubuntu/+archive/primary/+files/dante-server_1.4.2+dfsg-2build1_amd64.deb
dpkg -i dante-server_1.4.2+dfsg-2build1_amd64.deb
echo '
logoutput: syslog /var/log/danted.log
internal: eth0 port = 1080
external: eth0
 
socksmethod: username
user.privileged: root
user.unprivileged: nobody
 
client pass {
    from: 0.0.0.0/0 to: 0.0.0.0/0
    log: error
}
 
socks pass {
    from: 0.0.0.0/0 to: 0.0.0.0/0
    command: connect
    log: error
    socksmethod: username
}' > /etc/danted.conf

# basic ufw installation
sudo apt-get install ufw
sudo ufw status
# https://wiki.dieg.info/socks
sudo ufw allow ssh
sudo ufw allow proto tcp from any to any port 1080
sudo ufw status numbered
sudo ufw enable

sudo systemctl enable danted

sudo useradd --shell /usr/sbin/nologin av_socks && sudo passwd av_socks

So, thanks to bykvaadm for his feedback and support and to everybody else.

#linux

Also someone just bought us a coffee
- https://www.buymeacoffee.com/8oneCIN

Please consider supporting us for more quality content

Usually it takes several hours (to a month if it is about a competition) to write and does not pay well

And when people steal your content to put their refcodes in it, it's painful (

This is really interesting...their approach to separation is cool

Andrew NG released first 4 chapters of his new book
So far looks not really technical
- https://gallery.mailchimp.com/dc3a7ef4d750c0abfc19202a3/files/704291d2-365e-45bf-a9f5-719959dfe415/Ng_MLY01.pdf

#data_science

Given the current situation ... which post / guide would you like next?

DS / ML related (back log of hobby projects)! – 47
👍👍👍👍👍👍👍 69%

OpenVPN + Docker – 12
👍👍 18%

Dante proxy + Arubacloud + DigitalOcean + Vultr + Docker – 9
👍 13%

👥 68 people voted so far.

Useful Python abstractions / sugar / patterns

I already shared a book about patterns, which contains mostly high level / more complicated patters. But for writing ML code sometimes simple imperative function programming style is ok.

So - I will be posting about simple and really powerful python tips I am learning now.

This time I found out about map and filter, which are super useful for data preprocessing:

Map

items = [1, 2, 3, 4, 5]
squared = list(map(lambda x: x**2, items))

Filter

number_list = range(-5, 5)
less_than_zero = list(filter(lambda x: x < 0, number_list))
print(less_than_zero)

Also found this book - http://book.pythontips.com/en/latest/map_filter.html

#python
#data_science

A note on CDNs and protecting your website against censorship

TLDR
- https://goo.gl/47UqyZ
- Using a free / cheap CDN service can enable you to protect your domain hosted resource from censorship
- Unless CDN servers will be blocked (but I guess the CDN has more servers, than you, right?)

So, I host spark-in.me on Digital Ocean. And I do not want to move or start a CDN by myself. I read news, that Google abandoned some of its proxying tools because of such censorship events...interesting.

I knew that services like Cloudflare (**CDN**) forward your traffic somehow, but I was not sure what IP is actually seen by the user and whether all of the traffic is forwarded. Then I read their FAQ
- https://goo.gl/uHPLjW

It says

After a visitor's browser has done the initial DNS lookup, it begins making requests to retrieve the actual content of a website. These requests are directed to the IP address that was returned from the DNS lookup. Before Cloudflare, that address would have been 198.51.100.1. With Cloudflare as the authoritative nameserver, the new address is 203.0.113.1. Cloudflare’s data center at 203.0.113.1 will serve as much of your website as it can from its local storage, and ask your web server at 198.51.100.1 for any part of your website it doesn’t already have stored locally. The Cloudflare data center at 203.0.113.1 will then provide your complete website to the visitor, so the visitor never talks directly to your web server at 198.51.100.1.

So I tried their free-tier service (paid service starts from US$20-200, which is too steep) and it just works, though SSL certificates were issued ~90 mins after I changed my nameservers. It is as easy as:
- Backup your DNS settings somewhere
- Import to CloudFlare
- Change name servers in your domain registrar cabinet
- 90 mins and ... profit

Now I cannot see my direct DO server IP when I resolve my DNS:

$ dig +short spark-in.me
104.27.142.65
104.27.143.65

#internet
#security

Немного о баш скриптовании

Порою возникает необходимость записать какие-то данные в фай с сохранением переноса строк и отступов. сделать это можно несколькими способами.

1) с помощью echo

echo -e 'This is first string\nAnd this is second' > /path/to/file

Такой способ имеет только единственный плюс - он однострочник. В реальности он впринципе не читабельный.

Разбить этот однострочник можно, конечно, на несколько строк:

echo "
str1
   $variable - why not?
str N
"

И такой способ впринципе хорош до тех пор, пока вам не придется экранировать кавычки в тексте.

2) с помощью cat

cat > ceph.conf « EOF
[global]
mon_host = xx.xx.xx.xx:6789
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
EOF

В отличии от первого способа переменные по-прежнему интерпретируются, а кавычки экранировать не нужно.

#bash_tips_and_tricks

For windows users, that use their legacy machine as thin client to access Linux servers

Old habits die slowly. I use and old, but powerful Windows machine and so far doing everything on remote servers was ok, until I needed to commit to github using ssh agent forwarding.

But my key is stored locally and I do not want to use git bash or any windows based software, because it sucks. Also having a single source of truth on a remote Linux machine is better anyway. But I cannot store my key on the remote machine.

There is a solution - ssh-agent forwarding. In a nutshell:
- Install pageant, add your identity locally (.ppk private key file)
- Check allow agent forwarding in Putty
- Follow the below guides to check that all works
- profit

https://www.digitalocean.com/community/tutorials/how-to-use-pageant-to-streamline-ssh-key-authentication-with-putty

https://developer.github.com/v3/guides/using-ssh-agent-forwarding/

#linux

Dmytro's top4 solution on DS Bowl

https://www.kaggle.com/c/data-science-bowl-2018/discussion/55118

#deep_learning
#data_science

SOCK5 proxy configuration on Vultr

As of now, Vult was not (yet) blocked, probably because it is less known in the CIS. If you missed our Digital Ocean sock5 configuration guide, then you can follow this guide.

For us, both DO and Vulture work as of now.
- https://spark-in.me/post/vultr-sock5-proxy-server

You can use our referral links to create accounts
- https://m.do.co/c/6f8e77dddc23
- https://www.vultr.com/?ref=7402755

If you like the above guides, consider buying us a coffee
- https://www.buymeacoffee.com/8oneCIN

#linux
#digital_freedom

Readable list comprehensions in Python

My list and dictionary comprehensions usually look like s**t

https://gist.github.com/IaroslavR/7dcb54830242a22de1869f6fd05a8d7e

#python

DWT article on habrahabr.ru

DS Bowl article is live on habrahabr.ru
- https://habrahabr.ru/post/354040/

Please support us with your likes.

#data_science

Predictive models running on clientside?

Looks like soon you will have not only to use AdBlock but also restrict WebGL usage

https://youtu.be/Nc8kZABv-KE

#deep_learning