File formats for Machine Learning

Input File Formats
columnar data file - .orc,.petastorm,.parquet
Tabular File Formats - .csv, .xslx
Nested File Formats - .json, .tfrecords, .xml, .avro
ArrayBased File Formats - .npy
Hierarchical Data Formats - HDF5(.h5 or .hdf5) and NetCDF (.nc)

Model Formats
Tensorflow model —-> .pb file extension
Keras —> .h5
Scikit-learn —> .pkl
Older format —>.pmml
Apple developed —> .mlmodels store models in iOS.
Pytorch —> .pt

Content by Raja

Get ready for this week edition of DevSprint.

Let’s discuss the Blockchain this Sunday (tomorrow) morning 9AM IST.

You will be understanding basic and complete overview of Blockchain using python.

Meeting link will be share with you by end of the day.

random.shuffle() is an in-place operation that operates on any mutable sequence.

To shuffle in-line or to work from an immutable sequence, use random.sample() instead:

>>> s = 'hello world'
>>> ''.join(random.sample(s, k=len(s)))
'lld erlohwo'

The standard library shelve module provides a simple dict-like tool for saving data across python sessions.

>>> import shelve
>>> s = shelve.open('tmp.shelve')
>>> s['Raja']
'red'

It is a lightweight key/object data store.

Shelve is built on dbm and pickle, so it inherits their limitations.

When speed isn't a concern, I usually set writeback=True to avoid issues with mutable objects.

Algovis is a python library made for visualizing algorithms.


Currently, the library has these algorithms

-Sorting
Bubble Sort
Insertion Sort
Selection Sort
Merge Sort
Quick Sort

-Searching
Linear Search
Binary Search


Github: https://lnkd.in/gZShfJ3
Refer to the documentation for more info: https://lnkd.in/gUyybRi

🎺 pip 20.2 is out! 🎺

You can install it by running python -m pip install --upgrade pip.

The highlights for this release are:
———————
The beta of the next-generation dependency resolver is available
Faster installations from wheel files

Improved handling of wheels containing non-ASCII file contents

Faster pip list using parallelized network operations

Installed packages now contain metadata about whether they were directly requested by the user (PEP 376’s REQUESTED file)

The int() function is smarter than it looks:

‎>>> int('42')
‎42
‎>>> int('४२')
‎42
‎>>> int('٤٢')
‎42

What’s new in PyCharm 2020.2?

New support for GitHub Pull Requests, in-editor exception preview, and a revamped in-place refactoring experience.

Find out how you can take your productivity to the next level!

Check their official website for more details

📢 announcement 📢

╭━━━━━━╮
┃ ● ══ ┃
┃██████┃
┃ ┃
┃Launching ┃
┃iampython ┃
┃ ┃
┃ ┃
┃██████┃
┃ ○ ┃
╰━━━━━━╯


I have been working on iampython website from past few months.

This will be a one stop platform for python developers.

https://iampython.com

This is Beta version

Please have a look and signup.

You can ask your questions from this website.

Actual project goes live on 15th Aug with great content and so many features.

Let me know your feedback

Handling more than 200 transactions per second using python-rq

What is python-rq ?

RQ (Redis Queue) is a simple Python library for queueing jobs and processing them in the background with workers. It is backed by Redis and it is designed to have a low barrier to entry. It can be integrated in your web stack easily.

RQ requires Redis >= 3.0.0.

This is huge ...

I am ordered, mutable, and I don't allow duplicates. What am I?

A tuple

A dictionary

A set

An OrderedDict

A list

Python facts: OOP concept

super() can be used outside of a class definition.

>>> class A:
x = 10

>>> class B(A):
x = 12

>>> super(B, B).x
10

The super() call can't use the zero or one argument form because the class can't be deduced automatically from the context.

# Multiple Variable Assignment
a,b,*c=[1,2,3,4,5]
#print(a,b,c) ===> output : 1 2 [3, 4, 5]


#Swap two variables
a=1;b=5
a,b=b,a
#print(a,b) ===> output : 5 1


#Sum over every second element of a list
a=[1,2,3,4,5]
result=sum(a[1::2])
# print(result) ===> output : 6

content by Raja

Join Our Weekly DevSprint Meeting today @9PM IST (from now 45 minutes )
https://us02web.zoom.us/j/84643341101

Topic :
Commonly used python modules in realtime projects

Did you know?💡

The antigravity module originated in Google App Engine! Compared to the🐍Python 3's version it has a fly🦸‍♀️function that can even render the text of the comic in HTML.

The most popular Big Data tools are

Apache Spark,
Apache Kafka, and
Apache Hadoop/MapReduce, followed by Dask and Apache Hive.


We can also use these tools by python with different modules.

Here is the list of commonly used machine learning algorithms.
These algorithms can be applied to almost any data problem:

✅Linear Regression
✅Logistic Regression
✅Decision Tree
✅SVM
✅Naive Bayes
✅KNN
✅K-Means
✅Random Forest
✅Dimensionality Reduction Algorithms
✅Gradient Boosting algorithms
✔GBM
✔XGBoost
✔LightGBM
✔CatBoost

The overall market size for chatbots worldwide would be over $1.3B USD by 2024

Topic: DevSprint 011 - Building Next Generation Chatbots
Time: Aug 16, 2020 10:00 AM Mumbai, Kolkata, New Delhi

Join Zoom Meeting
https://us02web.zoom.us/j/84649074835


90 mins to go

Popular python based DeepLearning and Machine learning frameworks. https://iampython.com/resource-detail/top-python-deep-learning-frameworks-you-must-know/