arXiv digest 2024-12-12
📃 Computational Linguistics - top 5 papers of the day:
Filter-then-Generate: Large Language Models with Structure-Text Adapter for Knowledge Graph Completion
Summary:
We propose a novel instruction-tuning-based method, namely FtG, for knowledge graph completion.
💻 Code: https://github.com/LB0828/FtG}.
Read on arXiv
Olympus: A Universal Task Router for Computer Vision Tasks
Summary:
We introduce Olympus, a new approach that transforms Multimodal Large Language Models into a unified framework capable of handling a wide array of computer vision tasks.
💻 Code: https://github.com/yuanze-lin/Olympus_page
Read on arXiv
RuleArena: A Benchmark for Rule-Guided Reasoning with LLMs in Real-World Scenarios
Summary:
This paper introduces RuleArena, a novel and challenging benchmark designed to evaluate the ability of large language models (LLMs) to follow complex, real-world rules in reasoning.
💻 Code: https://github.com/skyriver-2000/RuleArena
Read on arXiv
Foundational Large Language Models for Materials Research
Summary:
We present a family of foundational models for materials science developed through continued pretraining of LLaMA models on an extensive corpus of materials literature and crystallographic data.
Read on arXiv
Mojito: Motion Trajectory and Intensity Control for Video Generation
Summary:
This paper introduces Mojito, a diffusion model that incorporates both motion trajectory and intensity control with high computational efficiency, generating motion patterns that closely match specified directions and intensities.
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
📃 Computational Linguistics - top 5 papers of the day:
Filter-then-Generate: Large Language Models with Structure-Text Adapter for Knowledge Graph Completion
Summary:
We propose a novel instruction-tuning-based method, namely FtG, for knowledge graph completion.
💻 Code: https://github.com/LB0828/FtG}.
Read on arXiv
Olympus: A Universal Task Router for Computer Vision Tasks
Summary:
We introduce Olympus, a new approach that transforms Multimodal Large Language Models into a unified framework capable of handling a wide array of computer vision tasks.
💻 Code: https://github.com/yuanze-lin/Olympus_page
Read on arXiv
RuleArena: A Benchmark for Rule-Guided Reasoning with LLMs in Real-World Scenarios
Summary:
This paper introduces RuleArena, a novel and challenging benchmark designed to evaluate the ability of large language models (LLMs) to follow complex, real-world rules in reasoning.
💻 Code: https://github.com/skyriver-2000/RuleArena
Read on arXiv
Foundational Large Language Models for Materials Research
Summary:
We present a family of foundational models for materials science developed through continued pretraining of LLaMA models on an extensive corpus of materials literature and crystallographic data.
Read on arXiv
Mojito: Motion Trajectory and Intensity Control for Video Generation
Summary:
This paper introduces Mojito, a diffusion model that incorporates both motion trajectory and intensity control with high computational efficiency, generating motion patterns that closely match specified directions and intensities.
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
arXiv digest 2024-12-12
🖼️ Computer Vision - top 5 papers of the day:
Feat2GS: Probing Visual Foundation Models with Gaussian Splatting
Summary:
We introduce Feat2GS, which readout 3D Gaussians attributes from VFM features extracted from unposed images, and investigate the ingredients that lead to a 3D aware VFM.
💻 Code: https://fanegg.github.io/Feat2GS/
Read on arXiv
Inference-Time Diffusion Model Distillation
Summary:
We introduce Distillation , a novel inference-time distillation framework that improves the denoising process in real-time without additional source data or fine-tuning.
💻 Code: https://github.com/geonyeong-park/inference_distillation
Read on arXiv
DisPose: Disentangling Pose Guidance for Controllable Human Image Animation
Summary:
DisPose disentangles the sparse skeleton pose in human image animation into motion field guidance and keypoint correspondence.
💻 Code: https://github.com/lihxxx/DisPose}{https://github.com/lihxxx/DisPose}.
Read on arXiv
Video Creation by Demonstration
Summary:
We present $\delta$-Diffusion, a self-supervised training approach that learns from unlabeled videos by conditional future frame prediction.
💻 Code: https://delta-diffusion.github.io/
Read on arXiv
OLA-VLM: Elevating Visual Perception in Multimodal LLMs with Auxiliary Embedding Distillation
Summary:
We propose OLA-VLM, the first approach distilling knowledge into the hidden representations from a set of target visual representations.
💻 Code: https://praeclarumjj3.github.io/ola_vlm/
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
🖼️ Computer Vision - top 5 papers of the day:
Feat2GS: Probing Visual Foundation Models with Gaussian Splatting
Summary:
We introduce Feat2GS, which readout 3D Gaussians attributes from VFM features extracted from unposed images, and investigate the ingredients that lead to a 3D aware VFM.
💻 Code: https://fanegg.github.io/Feat2GS/
Read on arXiv
Inference-Time Diffusion Model Distillation
Summary:
We introduce Distillation , a novel inference-time distillation framework that improves the denoising process in real-time without additional source data or fine-tuning.
💻 Code: https://github.com/geonyeong-park/inference_distillation
Read on arXiv
DisPose: Disentangling Pose Guidance for Controllable Human Image Animation
Summary:
DisPose disentangles the sparse skeleton pose in human image animation into motion field guidance and keypoint correspondence.
💻 Code: https://github.com/lihxxx/DisPose}{https://github.com/lihxxx/DisPose}.
Read on arXiv
Video Creation by Demonstration
Summary:
We present $\delta$-Diffusion, a self-supervised training approach that learns from unlabeled videos by conditional future frame prediction.
💻 Code: https://delta-diffusion.github.io/
Read on arXiv
OLA-VLM: Elevating Visual Perception in Multimodal LLMs with Auxiliary Embedding Distillation
Summary:
We propose OLA-VLM, the first approach distilling knowledge into the hidden representations from a set of target visual representations.
💻 Code: https://praeclarumjj3.github.io/ola_vlm/
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
arXiv digest 2024-12-12
🎵 Audio Processing - top 5 papers of the day:
Multimodal Music Generation with Explicit Bridges and Retrieval Augmentation
Summary:
We propose a novel multimodal music generation framework with explicit bridges of text and music for multimodality alignment.
💻 Code: https://github.com/wbs2788/VMB.
Read on arXiv
YingSound: Video-Guided Sound Effects Generation with Multi-modal Chain-of-Thought Controls
Summary:
We introduce YingSound, a foundation model designed for video-guided sound generation that supports high-quality audio generation in few-shot settings.
💻 Code: https://giantailab.github.io/yingsound/}
Read on arXiv
Interpreting Graphic Notation with MusicLDM: An AI Improvisation of Cornelius Cardew's Treatise
Summary:
This work presents a novel method for composing and improvising music inspired by Cornelius Cardew's Treatise, using AI to bridge graphic notation and musical expression.
Read on arXiv
Complex-Cycle-Consistent Diffusion Model for Monaural Speech Enhancement
Summary:
In this paper, we present a novel diffusion model-based monaural speech enhancement method.
Read on arXiv
Speech-Forensics: Towards Comprehensive Synthetic Speech Dataset Establishment and Analysis
Summary:
We propose the Speech-Forensics dataset by extensively covering authentic, synthetic, and partially forged speech samples that include multiple segments synthesized by different high-quality algorithms.
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
🎵 Audio Processing - top 5 papers of the day:
Multimodal Music Generation with Explicit Bridges and Retrieval Augmentation
Summary:
We propose a novel multimodal music generation framework with explicit bridges of text and music for multimodality alignment.
💻 Code: https://github.com/wbs2788/VMB.
Read on arXiv
YingSound: Video-Guided Sound Effects Generation with Multi-modal Chain-of-Thought Controls
Summary:
We introduce YingSound, a foundation model designed for video-guided sound generation that supports high-quality audio generation in few-shot settings.
💻 Code: https://giantailab.github.io/yingsound/}
Read on arXiv
Interpreting Graphic Notation with MusicLDM: An AI Improvisation of Cornelius Cardew's Treatise
Summary:
This work presents a novel method for composing and improvising music inspired by Cornelius Cardew's Treatise, using AI to bridge graphic notation and musical expression.
Read on arXiv
Complex-Cycle-Consistent Diffusion Model for Monaural Speech Enhancement
Summary:
In this paper, we present a novel diffusion model-based monaural speech enhancement method.
Read on arXiv
Speech-Forensics: Towards Comprehensive Synthetic Speech Dataset Establishment and Analysis
Summary:
We propose the Speech-Forensics dataset by extensively covering authentic, synthetic, and partially forged speech samples that include multiple segments synthesized by different high-quality algorithms.
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
arXiv digest 2024-12-12
🎛 Multimedia and Multimodal Learning - top 5 papers of the day:
MS2Mesh-XR: Multi-modal Sketch-to-Mesh Generation in XR Environments
Summary:
A novel multi-modal sketch-to-mesh generation pipeline that enables users to create realistic 3D objects in extended reality (XR) environments using hand-drawn sketches assisted by voice inputs.
💻 Code: https://yueqiu0911.github.io/MS2Mesh-XR/
Read on arXiv
Multimodal Music Generation with Explicit Bridges and Retrieval Augmentation
Summary:
We propose a novel multimodal music generation framework with explicit bridges of text and music for multimodality alignment.
💻 Code: https://github.com/wbs2788/VMB.
Read on arXiv
Lyra: An Efficient and Speech-Centric Framework for Omni-Cognition
Summary:
We introduce Lyra, an efficient MLLM that enhances multimodal abilities, including advanced long-speech comprehension, sound understanding, cross-modality efficiency, and seamless speech interaction.
Read on arXiv
Representing Long Volumetric Video with Temporal Gaussian Hierarchy
Summary:
We propose a novel 4D representation, named Temporal Gaussian Hierarchy, to compactly model long volumetric videos.
💻 Code: https://zju3dv.github.io/longvolcap
Read on arXiv
YingSound: Video-Guided Sound Effects Generation with Multi-modal Chain-of-Thought Controls
Summary:
We introduce YingSound, a foundation model designed for video-guided sound generation that supports high-quality audio generation in few-shot settings.
💻 Code: https://giantailab.github.io/yingsound/}
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
🎛 Multimedia and Multimodal Learning - top 5 papers of the day:
MS2Mesh-XR: Multi-modal Sketch-to-Mesh Generation in XR Environments
Summary:
A novel multi-modal sketch-to-mesh generation pipeline that enables users to create realistic 3D objects in extended reality (XR) environments using hand-drawn sketches assisted by voice inputs.
💻 Code: https://yueqiu0911.github.io/MS2Mesh-XR/
Read on arXiv
Multimodal Music Generation with Explicit Bridges and Retrieval Augmentation
Summary:
We propose a novel multimodal music generation framework with explicit bridges of text and music for multimodality alignment.
💻 Code: https://github.com/wbs2788/VMB.
Read on arXiv
Lyra: An Efficient and Speech-Centric Framework for Omni-Cognition
Summary:
We introduce Lyra, an efficient MLLM that enhances multimodal abilities, including advanced long-speech comprehension, sound understanding, cross-modality efficiency, and seamless speech interaction.
Read on arXiv
Representing Long Volumetric Video with Temporal Gaussian Hierarchy
Summary:
We propose a novel 4D representation, named Temporal Gaussian Hierarchy, to compactly model long volumetric videos.
💻 Code: https://zju3dv.github.io/longvolcap
Read on arXiv
YingSound: Video-Guided Sound Effects Generation with Multi-modal Chain-of-Thought Controls
Summary:
We introduce YingSound, a foundation model designed for video-guided sound generation that supports high-quality audio generation in few-shot settings.
💻 Code: https://giantailab.github.io/yingsound/}
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
arXiv digest 2024-12-13
📃 Computational Linguistics - top 5 papers of the day:
DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding
Summary:
We present DeepSeek-VL2, an advanced series of large Mixture-of-Experts (MoE) Vision-Language Models that significantly improves upon its predecessor, DeepSeeking-VL1.
💻 Code: https://github.com/deepseek-ai/DeepSeek-VL2.
Read on arXiv
RETQA: A Large-Scale Open-Domain Tabular Question Answering Dataset for Real Estate Sector
Summary:
We introduce RETQA, the first large-scale open-domain Chinese Tabular Question Answering dataset for Real Estate.
💻 Code: https://github.com/jensen-w/RETQA}.
Read on arXiv
Semi-IIN: Semi-supervised Intra-inter modal Interaction Learning Network for Multimodal Sentiment Analysis
Summary:
We propose a Semi-supervised Intra-inter modal Interaction Learning Network for multimodal sentiment analysis.
💻 Code: https://github.com/flow-ljh/Semi-IIN.
Read on arXiv
Can LLMs Convert Graphs to Text-Attributed Graphs?
Summary:
We propose a novel method named Topology-Aware Node description Synthesis (TANS) which leverages large language models (LLMs) to automatically convert existing graphs into text-attributed graphs.
💻 Code: https://github.com/Zehong-Wang/TANS.
Read on arXiv
Interlocking-free Selective Rationalization Through Genetic-based Learning
Summary:
We present the first interlocking-free architecture for selective rationalization that does not require any learning overhead.
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
📃 Computational Linguistics - top 5 papers of the day:
DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding
Summary:
We present DeepSeek-VL2, an advanced series of large Mixture-of-Experts (MoE) Vision-Language Models that significantly improves upon its predecessor, DeepSeeking-VL1.
💻 Code: https://github.com/deepseek-ai/DeepSeek-VL2.
Read on arXiv
RETQA: A Large-Scale Open-Domain Tabular Question Answering Dataset for Real Estate Sector
Summary:
We introduce RETQA, the first large-scale open-domain Chinese Tabular Question Answering dataset for Real Estate.
💻 Code: https://github.com/jensen-w/RETQA}.
Read on arXiv
Semi-IIN: Semi-supervised Intra-inter modal Interaction Learning Network for Multimodal Sentiment Analysis
Summary:
We propose a Semi-supervised Intra-inter modal Interaction Learning Network for multimodal sentiment analysis.
💻 Code: https://github.com/flow-ljh/Semi-IIN.
Read on arXiv
Can LLMs Convert Graphs to Text-Attributed Graphs?
Summary:
We propose a novel method named Topology-Aware Node description Synthesis (TANS) which leverages large language models (LLMs) to automatically convert existing graphs into text-attributed graphs.
💻 Code: https://github.com/Zehong-Wang/TANS.
Read on arXiv
Interlocking-free Selective Rationalization Through Genetic-based Learning
Summary:
We present the first interlocking-free architecture for selective rationalization that does not require any learning overhead.
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
arXiv digest 2024-12-13
🖼️ Computer Vision - top 5 papers of the day:
TSGaussian: Semantic and Depth-Guided Target-Specific Gaussian Splatting from Sparse Views
Summary:
We introduce a novel framework that combines semantic constraints with depth priors to avoid geometry degradation in challenging novel view synthesis tasks.
💻 Code: https://github.com/leon2000-ai/TSGaussian.
Read on arXiv
DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding
Summary:
We present DeepSeek-VL2, an advanced series of large Mixture-of-Experts (MoE) Vision-Language Models that significantly improves upon its predecessor, DeepSeeking-VL1.
💻 Code: https://github.com/deepseek-ai/DeepSeek-VL2.
Read on arXiv
EnvPoser: Environment-aware Realistic Human Motion Estimation from Sparse Observations with Uncertainty Modeling
Summary:
We propose a new method, EnvPoser, that employs a two-stage framework to perform full-body motion estimation using sparse tracking signals and pre-scanned environment from VR devices.
Read on arXiv
Building a Multi-modal Spatiotemporal Expert for Zero-shot Action Recognition with CLIP
Summary:
We propose a novel CLIP-based framework to comprehend multi-modal spatiotemporal dynamics synergistically.
💻 Code: https://github.com/Mia-YatingYu/STDD.
Read on arXiv
UniMed-CLIP: Towards a Unified Image-Text Pretraining Paradigm for Diverse Medical Imaging Modalities
Summary:
We introduce a large-scale, open-source multi-modal medical dataset comprising over 5.3 million image-text pairs across six diverse imaging modalities.
💻 Code: https://github.com/mbzuai-oryx/UniMed-CLIP
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
🖼️ Computer Vision - top 5 papers of the day:
TSGaussian: Semantic and Depth-Guided Target-Specific Gaussian Splatting from Sparse Views
Summary:
We introduce a novel framework that combines semantic constraints with depth priors to avoid geometry degradation in challenging novel view synthesis tasks.
💻 Code: https://github.com/leon2000-ai/TSGaussian.
Read on arXiv
DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding
Summary:
We present DeepSeek-VL2, an advanced series of large Mixture-of-Experts (MoE) Vision-Language Models that significantly improves upon its predecessor, DeepSeeking-VL1.
💻 Code: https://github.com/deepseek-ai/DeepSeek-VL2.
Read on arXiv
EnvPoser: Environment-aware Realistic Human Motion Estimation from Sparse Observations with Uncertainty Modeling
Summary:
We propose a new method, EnvPoser, that employs a two-stage framework to perform full-body motion estimation using sparse tracking signals and pre-scanned environment from VR devices.
Read on arXiv
Building a Multi-modal Spatiotemporal Expert for Zero-shot Action Recognition with CLIP
Summary:
We propose a novel CLIP-based framework to comprehend multi-modal spatiotemporal dynamics synergistically.
💻 Code: https://github.com/Mia-YatingYu/STDD.
Read on arXiv
UniMed-CLIP: Towards a Unified Image-Text Pretraining Paradigm for Diverse Medical Imaging Modalities
Summary:
We introduce a large-scale, open-source multi-modal medical dataset comprising over 5.3 million image-text pairs across six diverse imaging modalities.
💻 Code: https://github.com/mbzuai-oryx/UniMed-CLIP
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
arXiv digest 2024-12-13
🎵 Audio Processing - top 5 papers of the day:
CSL-L2M: Controllable Song-Level Lyric-to-Melody Generation Based on Conditional Transformer with Fine-Grained Lyric and Musical Controls
Summary:
We propose CSL-L2M, a controllable song-level lyric-to-melody generation method based on an in-attention Transformer decoder with fine-grained lyric and musical controls.
💻 Code: https://lichaiustc.github.io/CSL-L2M/.
Read on arXiv
CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models
Summary:
We present a scalable streaming speech synthesis model, CosyVoice 2, which incorporates comprehensive and systematic optimizations.
💻 Code: https://funaudiollm.github.io/cosyvoice2.
Read on arXiv
SonicBoom: Contact Localization Using Array of Microphones
Summary:
We introduce SonicBoom, a holistic hardware and learning pipeline that enables contact localization through an array of contact microphones
Read on arXiv
SILA: Signal-to-Language Augmentation for Enhanced Control in Text-to-Audio Generation
Summary:
We introduce a novel yet simple approach to generate sound effects with control over key acoustic parameters such as loudness, pitch, reverb, fade, brightness, noise and duration.
💻 Code: https://sonalkum.github.io/SILA/
Read on arXiv
Enhanced Speech Emotion Recognition with Efficient Channel Attention Guided Deep CNN-BiLSTM Framework
Summary:
We propose a lightweight speech emotion recognition architecture that integrates attention-based local feature blocks (ALFBs) to capture high-level relevant feature vectors from speech signals.
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
🎵 Audio Processing - top 5 papers of the day:
CSL-L2M: Controllable Song-Level Lyric-to-Melody Generation Based on Conditional Transformer with Fine-Grained Lyric and Musical Controls
Summary:
We propose CSL-L2M, a controllable song-level lyric-to-melody generation method based on an in-attention Transformer decoder with fine-grained lyric and musical controls.
💻 Code: https://lichaiustc.github.io/CSL-L2M/.
Read on arXiv
CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models
Summary:
We present a scalable streaming speech synthesis model, CosyVoice 2, which incorporates comprehensive and systematic optimizations.
💻 Code: https://funaudiollm.github.io/cosyvoice2.
Read on arXiv
SonicBoom: Contact Localization Using Array of Microphones
Summary:
We introduce SonicBoom, a holistic hardware and learning pipeline that enables contact localization through an array of contact microphones
Read on arXiv
SILA: Signal-to-Language Augmentation for Enhanced Control in Text-to-Audio Generation
Summary:
We introduce a novel yet simple approach to generate sound effects with control over key acoustic parameters such as loudness, pitch, reverb, fade, brightness, noise and duration.
💻 Code: https://sonalkum.github.io/SILA/
Read on arXiv
Enhanced Speech Emotion Recognition with Efficient Channel Attention Guided Deep CNN-BiLSTM Framework
Summary:
We propose a lightweight speech emotion recognition architecture that integrates attention-based local feature blocks (ALFBs) to capture high-level relevant feature vectors from speech signals.
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
arXiv digest 2024-12-13
🎛 Multimedia and Multimodal Learning - top 1 papers of the day:
EVOS: Efficient Implicit Neural Training via EVOlutionary Selector
Summary:
We propose EVOS, an efficient training paradigm for accelerating Implicit Neural Representation (INR).
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
🎛 Multimedia and Multimodal Learning - top 1 papers of the day:
EVOS: Efficient Implicit Neural Training via EVOlutionary Selector
Summary:
We propose EVOS, an efficient training paradigm for accelerating Implicit Neural Representation (INR).
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
arXiv digest 2024-12-14
📃 Computational Linguistics - top 5 papers of the day:
Towards Effective, Efficient and Unsupervised Social Event Detection in the Hyperbolic Space
Summary:
This work introduces an unsupervised framework, HyperSED, for social event detection in the hyperbolic space.
💻 Code: https://github.com/XiaoyanWork/HyperSED.
Read on arXiv
VisDoM: Multi-Document QA with Visually Rich Elements Using Multimodal Retrieval-Augmented Generation
Summary:
We introduce VisDoMBench, the first comprehensive benchmark designed to evaluate QA systems in multi-document settings with rich multimodal content, including tables, charts, and presentation slides.
Read on arXiv
Learning to Verify Summary Facts with Fine-Grained LLM Feedback
Summary:
We introduce FineSumFact, a large-scale dataset containing fine-grained factual feedback on summaries. We employ 10 distinct LLMs for diverse summary generation and Llama-3-70B-Instruct for feedback.
💻 Code: https://github.com/DISL-Lab/FineSumFact.
Read on arXiv
Tokens, the oft-overlooked appetizer: Large language models, the distributional hypothesis, and meaning
Summary:
We argue that large language models demonstrate that the Distributional Hypothesis (DM) is sufficient for reasonably human-like language performance, and that the emergence of human-meaningful linguistic units among tokens motivates linguistically-informed interventions in existing, linguistic-agnostic tokenization techniques.
Read on arXiv
Rethinking Chain-of-Thought from the Perspective of Self-Training
Summary:
We study Chain-of-Thought from the perspective of self-training and propose a novel framework that incorporates two key components: a task-specific prompt module designed to guide LLMs in generating high-quality initial reasoning processes, and an adaptive reasoning iteration module for progressively refining the reasoning process.
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
📃 Computational Linguistics - top 5 papers of the day:
Towards Effective, Efficient and Unsupervised Social Event Detection in the Hyperbolic Space
Summary:
This work introduces an unsupervised framework, HyperSED, for social event detection in the hyperbolic space.
💻 Code: https://github.com/XiaoyanWork/HyperSED.
Read on arXiv
VisDoM: Multi-Document QA with Visually Rich Elements Using Multimodal Retrieval-Augmented Generation
Summary:
We introduce VisDoMBench, the first comprehensive benchmark designed to evaluate QA systems in multi-document settings with rich multimodal content, including tables, charts, and presentation slides.
Read on arXiv
Learning to Verify Summary Facts with Fine-Grained LLM Feedback
Summary:
We introduce FineSumFact, a large-scale dataset containing fine-grained factual feedback on summaries. We employ 10 distinct LLMs for diverse summary generation and Llama-3-70B-Instruct for feedback.
💻 Code: https://github.com/DISL-Lab/FineSumFact.
Read on arXiv
Tokens, the oft-overlooked appetizer: Large language models, the distributional hypothesis, and meaning
Summary:
We argue that large language models demonstrate that the Distributional Hypothesis (DM) is sufficient for reasonably human-like language performance, and that the emergence of human-meaningful linguistic units among tokens motivates linguistically-informed interventions in existing, linguistic-agnostic tokenization techniques.
Read on arXiv
Rethinking Chain-of-Thought from the Perspective of Self-Training
Summary:
We study Chain-of-Thought from the perspective of self-training and propose a novel framework that incorporates two key components: a task-specific prompt module designed to guide LLMs in generating high-quality initial reasoning processes, and an adaptive reasoning iteration module for progressively refining the reasoning process.
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
arXiv digest 2024-12-14
🖼️ Computer Vision - top 5 papers of the day:
Video Diffusion Transformers are In-Context Learners
Summary:
This paper investigates a solution for enabling in-context capabilities of video diffusion transformers, with minimal tuning required for activation.
💻 Code: https://github.com/feizc/Video-In-Context}.
Read on arXiv
SegACIL: Solving the Stability-Plasticity Dilemma in Class-Incremental Semantic Segmentation
Summary:
We propose SegACIL, a novel continual learning method for semantic segmentation based on a linear closed-form solution.
💻 Code: https://github.com/qwrawq/SegACIL.
Read on arXiv
DSRC: Learning Density-insensitive and Semantic-aware Collaborative Representation against Corruptions
Summary:
We introduce the first comprehensive benchmark designed to evaluate the robustness of collaborative perception methods in the presence of natural corruptions typical of real-world environments.
💻 Code: https://github.com/Terry9a/DSRC.
Read on arXiv
Progressive Compression with Universally Quantized Diffusion Models
Summary:
We propose a new form of diffusion model with uniform noise in the forward process whose negative ELBO corresponds to the end-to-end compression cost using universal quantization.
Read on arXiv
Rebalanced Vision-Language Retrieval Considering Structure-Aware Distillation
Summary:
We propose a simple yet effective method to rebalance cross-modal matching by learning structure-preserved matching representations.
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
🖼️ Computer Vision - top 5 papers of the day:
Video Diffusion Transformers are In-Context Learners
Summary:
This paper investigates a solution for enabling in-context capabilities of video diffusion transformers, with minimal tuning required for activation.
💻 Code: https://github.com/feizc/Video-In-Context}.
Read on arXiv
SegACIL: Solving the Stability-Plasticity Dilemma in Class-Incremental Semantic Segmentation
Summary:
We propose SegACIL, a novel continual learning method for semantic segmentation based on a linear closed-form solution.
💻 Code: https://github.com/qwrawq/SegACIL.
Read on arXiv
DSRC: Learning Density-insensitive and Semantic-aware Collaborative Representation against Corruptions
Summary:
We introduce the first comprehensive benchmark designed to evaluate the robustness of collaborative perception methods in the presence of natural corruptions typical of real-world environments.
💻 Code: https://github.com/Terry9a/DSRC.
Read on arXiv
Progressive Compression with Universally Quantized Diffusion Models
Summary:
We propose a new form of diffusion model with uniform noise in the forward process whose negative ELBO corresponds to the end-to-end compression cost using universal quantization.
Read on arXiv
Rebalanced Vision-Language Retrieval Considering Structure-Aware Distillation
Summary:
We propose a simple yet effective method to rebalance cross-modal matching by learning structure-preserved matching representations.
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
arXiv digest 2024-12-14
🎵 Audio Processing - top 5 papers of the day:
VinTAGe: Joint Video and Text Conditioning for Holistic Audio Generation
Summary:
We introduce VinTAGe, a flow-based transformer model that jointly considers text and video to guide audio generation.
Read on arXiv
Efficient Adaptation of Multilingual Models for Japanese ASR
Summary:
We fine-tuning Whisper-Tiny to improve performance in Japanese.
Read on arXiv
Composers' Evaluations of an AI Music Tool: Insights for Human-Centred Design
Summary:
We present a study that explores the role of user-centred design in developing Generative AI (GenAI) tools for music composition
Read on arXiv
Audio-based Anomaly Detection in Industrial Machines Using Deep One-Class Support Vector Data Description
Summary:
Audio-based anomaly detection in industrial machines using Deep One-Class Support Vector Data Description
Read on arXiv
Hidden Echoes Survive Training in Audio To Audio Generative Instrument Models
Summary:
If imperceptible echoes are hidden in the training data, a wide variety of audio to audio architectures (differentiable digital signal processing (DDSP), Realtime Audio Variational autoEncoder (RAVE), and ``Dance Diffusion'') will reproduce these echoes in their outputs.
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
🎵 Audio Processing - top 5 papers of the day:
VinTAGe: Joint Video and Text Conditioning for Holistic Audio Generation
Summary:
We introduce VinTAGe, a flow-based transformer model that jointly considers text and video to guide audio generation.
Read on arXiv
Efficient Adaptation of Multilingual Models for Japanese ASR
Summary:
We fine-tuning Whisper-Tiny to improve performance in Japanese.
Read on arXiv
Composers' Evaluations of an AI Music Tool: Insights for Human-Centred Design
Summary:
We present a study that explores the role of user-centred design in developing Generative AI (GenAI) tools for music composition
Read on arXiv
Audio-based Anomaly Detection in Industrial Machines Using Deep One-Class Support Vector Data Description
Summary:
Audio-based anomaly detection in industrial machines using Deep One-Class Support Vector Data Description
Read on arXiv
Hidden Echoes Survive Training in Audio To Audio Generative Instrument Models
Summary:
If imperceptible echoes are hidden in the training data, a wide variety of audio to audio architectures (differentiable digital signal processing (DDSP), Realtime Audio Variational autoEncoder (RAVE), and ``Dance Diffusion'') will reproduce these echoes in their outputs.
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
arXiv digest 2024-12-14
🎛 Multimedia and Multimodal Learning - top 5 papers of the day:
UCDR-Adapter: Exploring Adaptation of Pre-Trained Vision-Language Models for Universal Cross-Domain Retrieval
Summary:
We propose UCDR-Adapter, which enhances pre-trained models with adapters and dynamic prompt generation through a two-phase training strategy.
💻 Code: https://github.com/fine68/UCDR2024
Read on arXiv
MambaPro: Multi-Modal Object Re-Identification with Mamba Aggregation and Synergistic Prompt
Summary:
We introduce a novel framework called MambaPro for multi-modal object Re-IDentification.
💻 Code: https://github.com/924973292/MambaPro.
Read on arXiv
VinTAGe: Joint Video and Text Conditioning for Holistic Audio Generation
Summary:
We introduce VinTAGe, a flow-based transformer model that jointly considers text and video to guide audio generation.
Read on arXiv
Patch-level Sounding Object Tracking for Audio-Visual Question Answering
Summary:
We present a new Patch-level Sounding Object Tracking method, which relies on visual motion information to identify salient visual patches with significant movements that are more likely to relate to sounding objects and questions.
Read on arXiv
Hidden Echoes Survive Training in Audio To Audio Generative Instrument Models
Summary:
If imperceptible echoes are hidden in the training data, a wide variety of audio to audio architectures (differentiable digital signal processing (DDSP), Realtime Audio Variational autoEncoder (RAVE), and ``Dance Diffusion'') will reproduce these echoes in their outputs.
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
🎛 Multimedia and Multimodal Learning - top 5 papers of the day:
UCDR-Adapter: Exploring Adaptation of Pre-Trained Vision-Language Models for Universal Cross-Domain Retrieval
Summary:
We propose UCDR-Adapter, which enhances pre-trained models with adapters and dynamic prompt generation through a two-phase training strategy.
💻 Code: https://github.com/fine68/UCDR2024
Read on arXiv
MambaPro: Multi-Modal Object Re-Identification with Mamba Aggregation and Synergistic Prompt
Summary:
We introduce a novel framework called MambaPro for multi-modal object Re-IDentification.
💻 Code: https://github.com/924973292/MambaPro.
Read on arXiv
VinTAGe: Joint Video and Text Conditioning for Holistic Audio Generation
Summary:
We introduce VinTAGe, a flow-based transformer model that jointly considers text and video to guide audio generation.
Read on arXiv
Patch-level Sounding Object Tracking for Audio-Visual Question Answering
Summary:
We present a new Patch-level Sounding Object Tracking method, which relies on visual motion information to identify salient visual patches with significant movements that are more likely to relate to sounding objects and questions.
Read on arXiv
Hidden Echoes Survive Training in Audio To Audio Generative Instrument Models
Summary:
If imperceptible echoes are hidden in the training data, a wide variety of audio to audio architectures (differentiable digital signal processing (DDSP), Realtime Audio Variational autoEncoder (RAVE), and ``Dance Diffusion'') will reproduce these echoes in their outputs.
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
arXiv digest 2024-12-15
📃 Computational Linguistics - top 5 papers of the day:
Smaller Language Models Are Better Instruction Evolvers
Summary:
We propose Instruction Complex-Aware IFD (IC-IFD), which introduces instruction complexity in the original IFD score to evaluate the effectiveness of instruction data more accurately.
💻 Code: https://github.com/HypherX/Evolution-Analysis}{https://github.com/HypherX/Evolution-Analysis}
Read on arXiv
CATER: Leveraging LLM to Pioneer a Multidimensional, Reference-Independent Paradigm in Translation Quality Evaluation
Summary:
This paper introduces CATER, a novel and fully prompt-driven framework for evaluating machine translation quality.
Read on arXiv
The Superalignment of Superhuman Intelligence with Large Language Models
Summary:
We discuss the concept of superalignment from the learning perspective and highlight some key research problems in the learning paradigm.
Read on arXiv
TrimLLM: Progressive Layer Dropping for Domain-Specific LLMs
Summary:
We develop TrimLLM based on the layer-wise specialization phenomenon we empirically observed and verified on contemporary LLMs.
Read on arXiv
Transliterated Zero-Shot Domain Adaptation for Automatic Speech Recognition
Summary:
We propose transliterated ZSDA to achieve consistent pre-training and fine-tuning labels, leading to maximum preservation of the pre-trained knowledge.
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
📃 Computational Linguistics - top 5 papers of the day:
Smaller Language Models Are Better Instruction Evolvers
Summary:
We propose Instruction Complex-Aware IFD (IC-IFD), which introduces instruction complexity in the original IFD score to evaluate the effectiveness of instruction data more accurately.
💻 Code: https://github.com/HypherX/Evolution-Analysis}{https://github.com/HypherX/Evolution-Analysis}
Read on arXiv
CATER: Leveraging LLM to Pioneer a Multidimensional, Reference-Independent Paradigm in Translation Quality Evaluation
Summary:
This paper introduces CATER, a novel and fully prompt-driven framework for evaluating machine translation quality.
Read on arXiv
The Superalignment of Superhuman Intelligence with Large Language Models
Summary:
We discuss the concept of superalignment from the learning perspective and highlight some key research problems in the learning paradigm.
Read on arXiv
TrimLLM: Progressive Layer Dropping for Domain-Specific LLMs
Summary:
We develop TrimLLM based on the layer-wise specialization phenomenon we empirically observed and verified on contemporary LLMs.
Read on arXiv
Transliterated Zero-Shot Domain Adaptation for Automatic Speech Recognition
Summary:
We propose transliterated ZSDA to achieve consistent pre-training and fine-tuning labels, leading to maximum preservation of the pre-trained knowledge.
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
arXiv digest 2024-12-15
🖼️ Computer Vision - top 5 papers of the day:
Reason-before-Retrieve: One-Stage Reflective Chain-of-Thoughts for Training-Free Zero-Shot Composed Image Retrieval
Summary:
We propose a novel training-free one-stage method, One-Stage Reflective Chain-of-Thought Reasoning for ZS-CIR (OSrCIR), which employs Multimodal Large Language Models to retain essential visual information in a single-stage reasoning process, eliminating the information loss seen in two-stage methods.
💻 Code: https://github.com/Pter61/osrcir2024/.
Read on arXiv
Benchmarking and Learning Multi-Dimensional Quality Evaluator for Text-to-3D Generation
Summary:
We propose a comprehensive benchmark for evaluating text-to-3D generation and propose a novel quality evaluator named HyperScore.
💻 Code: https://mate-3d.github.io/.
Read on arXiv
SHMT: Self-supervised Hierarchical Makeup Transfer via Latent Diffusion Models
Summary:
We propose a novel Self-supervised Hierarchical Makeup Transfer (SHMT) method via latent diffusion models.
💻 Code: https://github.com/Snowfallingplum/SHMT}.
Read on arXiv
Exploring Enhanced Contextual Information for Video-Level Object Tracking
Summary:
We propose a new video-level object tracking framework called MCITrack, which leverages Mamba's hidden states to continuously record and transmit extensive contextual information throughout the video stream.
💻 Code: https://github.com/kangben258/MCITrack.
Read on arXiv
GaussianProperty: Integrating Physical Properties to 3D Gaussians with LMMs
Summary:
We introduce GaussianProperty, a training-free framework that assigns physical properties of materials to 3D Gaussians.
💻 Code: https://Gaussian-Property.github.io}{this
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
🖼️ Computer Vision - top 5 papers of the day:
Reason-before-Retrieve: One-Stage Reflective Chain-of-Thoughts for Training-Free Zero-Shot Composed Image Retrieval
Summary:
We propose a novel training-free one-stage method, One-Stage Reflective Chain-of-Thought Reasoning for ZS-CIR (OSrCIR), which employs Multimodal Large Language Models to retain essential visual information in a single-stage reasoning process, eliminating the information loss seen in two-stage methods.
💻 Code: https://github.com/Pter61/osrcir2024/.
Read on arXiv
Benchmarking and Learning Multi-Dimensional Quality Evaluator for Text-to-3D Generation
Summary:
We propose a comprehensive benchmark for evaluating text-to-3D generation and propose a novel quality evaluator named HyperScore.
💻 Code: https://mate-3d.github.io/.
Read on arXiv
SHMT: Self-supervised Hierarchical Makeup Transfer via Latent Diffusion Models
Summary:
We propose a novel Self-supervised Hierarchical Makeup Transfer (SHMT) method via latent diffusion models.
💻 Code: https://github.com/Snowfallingplum/SHMT}.
Read on arXiv
Exploring Enhanced Contextual Information for Video-Level Object Tracking
Summary:
We propose a new video-level object tracking framework called MCITrack, which leverages Mamba's hidden states to continuously record and transmit extensive contextual information throughout the video stream.
💻 Code: https://github.com/kangben258/MCITrack.
Read on arXiv
GaussianProperty: Integrating Physical Properties to 3D Gaussians with LMMs
Summary:
We introduce GaussianProperty, a training-free framework that assigns physical properties of materials to 3D Gaussians.
💻 Code: https://Gaussian-Property.github.io}{this
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
arXiv digest 2024-12-15
🎵 Audio Processing - top 5 papers of the day:
Efficient Whisper on Streaming Speech
Summary:
We introduce Whisper-T, an innovative framework combining both model and system-level optimizations for streaming speech.
Read on arXiv
Sonicmesh: Enhancing 3D Human Mesh Reconstruction in Vision-Impaired Environments With Acoustic Signals
Summary:
A novel approach combining acoustic signals with RGB images to reconstruct 3D human mesh in challenging environments with poor lighting, privacy concerns, or occlusions.
Read on arXiv
Transliterated Zero-Shot Domain Adaptation for Automatic Speech Recognition
Summary:
We propose transliterated ZSDA to achieve consistent pre-training and fine-tuning labels, leading to maximum preservation of the pre-trained knowledge.
Read on arXiv
Imagined Speech State Classification for Robust Brain-Computer Interface
Summary:
This study examines the effectiveness of traditional machine learning classifiers versus deep learning models for detecting the imagined speech using electroencephalogram data.
Read on arXiv
Hanprome: Modified Hangeul for Expression of foreign language pronunciation
Summary:
Hangeul can be used as a kind of phonetic symbol.
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
🎵 Audio Processing - top 5 papers of the day:
Efficient Whisper on Streaming Speech
Summary:
We introduce Whisper-T, an innovative framework combining both model and system-level optimizations for streaming speech.
Read on arXiv
Sonicmesh: Enhancing 3D Human Mesh Reconstruction in Vision-Impaired Environments With Acoustic Signals
Summary:
A novel approach combining acoustic signals with RGB images to reconstruct 3D human mesh in challenging environments with poor lighting, privacy concerns, or occlusions.
Read on arXiv
Transliterated Zero-Shot Domain Adaptation for Automatic Speech Recognition
Summary:
We propose transliterated ZSDA to achieve consistent pre-training and fine-tuning labels, leading to maximum preservation of the pre-trained knowledge.
Read on arXiv
Imagined Speech State Classification for Robust Brain-Computer Interface
Summary:
This study examines the effectiveness of traditional machine learning classifiers versus deep learning models for detecting the imagined speech using electroencephalogram data.
Read on arXiv
Hanprome: Modified Hangeul for Expression of foreign language pronunciation
Summary:
Hangeul can be used as a kind of phonetic symbol.
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
arXiv digest 2024-12-15
🎛 Multimedia and Multimodal Learning - top 2 papers of the day:
Multimodal Class-aware Semantic Enhancement Network for Audio-Visual Video Parsing
Summary:
We propose a multimodal class-aware semantic Enhancement network for audio-visual video parsing.
Read on arXiv
Provably Secure Robust Image Steganography via Cross-Modal Error Correction
Summary:
We propose a high-quality, provably secure, and robust image steganography method based on state-of-the-art autoregressive (AR) image generation models using VQ tokenizers.
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates
🎛 Multimedia and Multimodal Learning - top 2 papers of the day:
Multimodal Class-aware Semantic Enhancement Network for Audio-Visual Video Parsing
Summary:
We propose a multimodal class-aware semantic Enhancement network for audio-visual video parsing.
Read on arXiv
Provably Secure Robust Image Steganography via Cross-Modal Error Correction
Summary:
We propose a high-quality, provably secure, and robust image steganography method based on state-of-the-art autoregressive (AR) image generation models using VQ tokenizers.
Read on arXiv
Disclaimer: paper selection is AI curated.
by @deeplearning_updates