On evaluating the faithfulness of explanations like attention and gradients -- the idea is to retrain a model by removing allegedly important tokens and measure the drop in performance, inspired from ROAR in vision. More faithful an explanation more the drop #NLProc
Can you trust attention explanations in NLProc? What about other explanations?
arxiv.org/abs/2110.08412
#مقاله

❇️ @AI_Python
🔰 @DLeX_Apply

New PhD position open on *eco-sustainable large language models* in my group at the #SapienzaNLP University of #Rome
+ 12 months at Multilinguality at your fingertips.
with tight deadline (October 27th) to join 7 other new students this year! Contact me! #NLProc #languagemodels #neurosymbolic #PhD
https://phd.uniroma1.it/web/concorso37pon.aspx?s=&i=3514&m=&l=EN&p=&a=

nlp.uniroma1.it

❇️ @AI_Python
🔰 @DLeX_Apply

New ICLR paper: Autoregressive Diffusion Models. ARDMs generate variables in an arbitrary order, one by one, and are able to upscale variables, such as the bit values of a pixel. For an #NLProc audience, this will remind you of BERT, and even more so XLNet.

Generate data in random order? A masked language models as generative model?
All this and more in "Autoregressive Diffusion Models"

#مقاله #
✴️ @AI_Python