In python 3.9, PEP-616 introduced str.removeprefix and str.removesuffix methods:

'abcd'.removeprefix('ab')
# 'cd'

'abcd'.removeprefix('fg')
# 'abcd'

The implementation is simple (it's implemented on C, of course, but the idea is the same):

def removeprefix(self: str, prefix: str) -> str:
if self.startswith(prefix):
return self[len(prefix):]
return self

​​Exploring Transfer Learning with T5: the Text-To-Text Transfer Transformer

tl;dr:
- 11 billion parameters
- encoder-decoder models generally outperformed “decoder-only” language models
- fill-in-the-blank-style denoising objectives worked best;
- the most important factor was the computational cost;
- training on in-domain data can be beneficial but that pre-training on smaller datasets can lead to detrimental overfitting;
- multitask learning could be close to competitive with a pre-train-then-fine-tune approach but requires carefully choosing how often the model is trained on each task

The model can be fine-tuned on smaller labeled datasets, often resulting in (far) better performance than training on the labeled data alone.
Present a large-scale empirical survey to determine which transfer learning techniques work best and apply these insights at scale to create a new model that we call the T5. Also, introduce a new open-source pre-training dataset, called the Colossal Clean Crawled Corpus (C4).

The T5 model, pre-trained on C4, achieves SOTA results on many NLP benchmarks while being flexible enough to be fine-tuned to a variety of important downstream tasks.


blog post: https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html
paper: https://arxiv.org/abs/1910.10683
github (with pre-trained models): https://github.com/google-research/text-to-text-transfer-transformer
colab notebook: https://colab.research.google.com/github/google-research/text-to-text-transfer-transformer/blob/master/notebooks/t5-trivia.ipynb

#nlp #transformer #t5

Interesting paper bout reproducibility in AI/ML from Dr. Edward Raff is a Chief Scientist at Booz Allen Hamilton. He analyzed 255 papers, and successfully reproduce 162 from them.

A 62% success rate is higher than many meta-analyses from other sciences, and I suspect my 62% number is lower than reality

Interesting facts:
1. Having fewer equations per page makes a paper more reproducible.
2. Empirical papers may be more reproducible than theory-oriented papers.
3. Sharing code is not a panacea
4. Having detailed pseudo code is just as reproducible as having no pseudo code.
5. Creating simplified example problems do not appear to help with reproducibility.
6: Please, check your email (papers of people who answer on emails is more reproducible)

https://thegradient.pub/independently-reproducible-machine-learning/

Leave a comment on how do you feel about this.

https://github.com/ignani/PythonMLDataScience

Have just uploaded some samples. Will keep adding as and when I get some free time. You can use it for learning.

Let me know what view. 😊

Nextstrain build for novel coronavirus (nCoV)

https://github.com/nextstrain/ncov

For all those who are looking for help on learning Python, or looking for practice projects to work on... check out this topic https://t.me/python_programmers_club/121484

Data structures are used to organize and store data efficiently in a computer so that they can be accessed and modified efficiently. Examples of data structures include arrays, linked lists, stacks, queues, trees, and graphs.

Algorithms are step-by-step procedures for solving a problem or performing a task. Algorithms can be implemented using data structures to improve their efficiency and performance. Examples of algorithms include sorting algorithms (such as quicksort and mergesort), search algorithms (such as binary search), and graph algorithms (such as Dijkstra's shortest path algorithm).

Both data structures and algorithms are fundamental concepts in computer science and are widely used in various applications, such as databases, operating systems, computer networks, and software engineering. The choice of data structure and algorithm can greatly impact the efficiency and performance of a software system.

In the Python programming language, there are several built-in data structures such as lists, tuples, sets, and dictionaries. Lists are ordered collections of elements and can be modified, while tuples are ordered, immutable collections. Sets are unordered collections of unique elements, and dictionaries are unordered collections of key-value pairs.

Python also has a rich set of libraries for implementing various algorithms, including NumPy and SciPy for numerical computing, and pandas for data analysis.

For sorting, the "sorted" function can be used, which returns a sorted list, while the "sort" method can be used to sort a list in place. For searching, the "in" operator can be used to check if an element is in a list or a dictionary, or the "index" method can be used to find the first occurrence of an element in a list.

In addition, Python has a large community of users and developers who have contributed many open-source packages that implement a wide variety of algorithms, making it easier for developers to incorporate these algorithms into their own projects.

To learn data structures and algorithms in Python, you can follow these steps:

1. Start with the basics: Learn about the most common data structures, such as arrays, linked lists, stacks, queues, trees, and graphs. Learn how to implement them in Python and understand their time and space complexities.

2. Study algorithms: Study the most common algorithms for searching, sorting, and traversing data structures. Understand their time and space complexities and the trade-offs between different algorithms.

3. Practice, practice, practice: The more you practice implementing data structures and algorithms, the better you will get at it. You can start by solving problems on websites like LeetCode and HackerRank, or by working on small projects of your own.

4. Read and learn from others: Read articles and blogs written by experts in the field, and learn from their experiences and insights. Follow the work of other Python developers on Github, and see how they use data structures and algorithms in their projects.

Practice projects to consider:

1. Implement a basic search engine: Read a set of documents and build an index of keywords. Then, implement a search function that returns a list of documents that match the query.

2. Build a recommendation system: Read a set of user-item interactions and build a recommendation system that suggests items to users based on their past behavior.

3. Create a data analysis tool: Read a large dataset and implement a tool that performs various analyses, such as calculating summary statistics, visualizing distributions, and identifying patterns and correlations.

4. Implement a graph algorithm: Study a graph algorithm such as Dijkstra's shortest path algorithm, and implement it in Python. Then, test it on real-world graphs to see how it performs.

PyTip for the day: When working with lists in Python, consider using list comprehensions instead of for loops to perform operations on the list elements. List comprehensions are more concise, readable, and often faster than using traditional for loops.

For example, instead of using a for loop to create a new list that contains the squared values of the elements in an existing list, you can use a list comprehension like this:

> numbers = 1, 2, 3, 4, 5
> squarednumbers = [num ** 2 for num in numbers]

This will create a new list squarednumbers containing the squared values of the elements in numbers.

PyTip for the day: List comprehension, with more examples:

Check out previous tip to begin with. Use list comprehension to create a new list based on an existing list with minimal code.

List comprehension is a concise and readable way to create a new list by transforming or filtering an existing list. It can make your code more efficient and easier to read. Here's an example:

 Create a new list that contains the squares of numbers from 1 to 10
squares = [i**2 for i in range(1, 11)]

# Print the squares
print(squares)  # Output: [1, 4, 9, 16, 25, 36, 49, 64, 81, 100]

In this example, we use a for loop and the range() function to create a list of numbers from 1 to 10, and then use list comprehension to create a new list that contains the squares of these numbers.

List comprehension can also be used for filtering elements from an existing list, like this:

 Filter out the odd numbers from a list of numbers
numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
even_numbers = [n for n in numbers if n % 2 == 0]

# Print the even numbers
print(even_numbers)  # Output: [2, 4, 6, 8, 10]

In this example, we use list comprehension and the modulo operator to create a new list that contains only the even numbers from the original list of numbers.

List comprehension is a powerful and versatile feature of Python, and can save you a lot of time and effort when working with lists.

PyTip for the day: `with` statement
Use the "with" statement to automatically close files after reading or writing data.

When working with files in Python, it's important to remember to close the file after you're done with it. If you don't close the file, it can lead to data loss or other errors.

One way to ensure that your files are always closed properly is to use the "with" statement. The "with" statement automatically closes the file for you when you're done with it, even if an error occurs. Here's an example:
# Open a file and read its contents using the "with" statement
with open('myfile.txt', 'r') as file:
data = file.read()
print(data)

In this example, we use the "with" statement to open a file named "myfile.txt" for reading. We read the contents of the file using the file.read() method and print it to the console. When the block of code inside the "with" statement is finished executing, the file is automatically closed.

You can also use the "with" statement for writing to files, like this:
# Open a file and write some data to it using the "with" statement
with open('myfile.txt', 'w') as file:
file.write('Hello, world!')

In this example, we use the "with" statement to open a file named "myfile.txt" for writing. We write the string "Hello, world!" to the file using the file.write() method. When the block of code inside the "with" statement is finished executing, the file is automatically closed.

Using the "with" statement is a good habit to get into when working with files in Python, as it helps to ensure that your files are always closed properly and that your data is safe.