Q: Briefly describe two similarities and two dissimilarities of ConvNeXt architecture, compared to ResNet.

A:
Similarities:
1. They are both only utilizing convolution operations for feature extraction (no attention needed)
2. They both utilize residual skip connections in their building blocks
3. They have the same number of "stages", in which operations are performed on the same spatial size, followed by trainable down-sampling, implemented with Conv layer

Differences:
1. Inverted bottleneck - in ConvNext first spatial mixing is performed with depth-wise convolution, then channel mixing using 1x1 convolutions vs in ResNet first squeezing # of channels using 1x1 convolution and then performing combined channel-spatial mixing using 3x3 convolution.

2. Patchify stem - in its stem ConvNext utilizes approach similar to ViT architecture, splitting the images into 4x4 patches with stride 4, effectively converting each of these patches into independent embeddings using convolution vs ResNet uses 7x7 convolution with stride 2 and 2x2 maxpool with stride 2 to achieve 4x down-sampling of the features with respect to the original image, and each "pixel" of the feature map has information about neighboring pixels.

3. LayerNorm instead of BatchNorm - in BN, features are normalized across samples, leading to randomized behavior, and feature statistics are only measured during training, which leads to different behavior during inference, while LayerNorm has similar behavior across stages and does not depend on the input.

4. Less Normalizations and Activations - in ConvNext, normalization is only performed after non-unit-size convolutions (Stem, Inter-stage downsampling, inside Bottleneck blocks), while in ResNet normalization is performed after each convolution. Also, only 1 activation is used inside ConvNext block, between channel mixing stages, against the starnart convention of using activation after every convolution, mimicking the structure of Swin block

​🧑‍🎓 Leetcode по ML/DS

Думаю, все знают про leetcode, с помощью которого можно отлично натаскаться на алгоритмические задачки.
Нашли аналогичный сервис по ML/DS задачкам, на котором можно попрактиковаться в решении задач по SQL, Python, Теории вероятностей и статистике. В нём собраны задачки, которые спрашивают топовых компаниях вроде Tesla/Twitter/Facebook/Linkedin и т.д.

Отличная штука для того, чтобы попрактиковаться перед собеседованием 🙂

https://datalemur.com/questions

#algo #code

#ml #competition

#dl #recommender

Face reveal: https://youtu.be/0ZU-FtLO4Fw?t=2140

Рассказываю про нейросетевое ранжирование для рекомендаций.

После доклада за кулисами обсудили РЛ, графовые сетки, bias'ы, трансформеры для персонализации, академию, индустрию, роблокс, рекомендательные ленты, next item prediction, movielens, как делать надо и как не надо, симуляции пользователей, языковые модели, объяснения рекомендаций, манипуляции вместо рекомендаций, trust bias, фидбек луп.

P.S: Подписчики на митапе зашеймили за отсутствие постов на канале. Заверил, что это временно :)

#mlops #courses