Amir Feder

Columbia University

I am a postdoctoral fellow at the Columbia Data Science Institute, working with David Blei on causal inference and natural language processing. I am also a visiting faculty researcher at Google Research, where I work on methods that leverage causality for building factual and consistent language models. 


My research seeks to develop methods that integrate causality into natural language processing, and use them to build linguistically-informed algorithms for predicting and understanding human behavior. Deep language models are powerful prediction machines that seem to be improving on a daily basis. They have reached super-human performance on tasks that seemed impossible only a few years ago. Unfortunately, this improvement has resulted in unimaginably large models that often do not take into account the world they are operating in, and are both brittle and hard to interpret. My goal is to address such concerns using causal inference, to allow us to use these models to better predict and understand human behavior. Through the paradigm of causal machine learning, I aim to build bridges between machine learning and the social sciences. 


Before joining Columbia, I received my PhD from the Technion,  where I was advised by Roi Reichart and worked closely with Uri Shalit. In a previous (academic) life, I was an economics, statistics and history student at Tel Aviv University,  the Hebrew University of Jerusalem and Northwestern University.


email: amir.feder at columbia dot edu

[google scholar] [semantic scholar] [dblp] [github] 

Selected Publications (*=equal contribution)


CausaLM: Causal model explanation through counterfactual language models

Amir Feder, Nadav Oved, Uri Shalit, Roi Reichart

Computational Linguistics (CL),  2021 [arxiv] [code] [data] [pdf]


On Calibration and Out-of-domain Generalization

Yoav Wald*, Amir Feder*, Daniel Greenfeld, Uri Shalit

Advances in neural information processing systems (NeurIPS) 2021  [arxiv] [pdf]


In the Eye of the Beholder: Robust Prediction with Causal User Modeling

Amir Feder, Guy Horowitz, Yoav Wald, Roi Reichart, and Nir Rosenfeld

Advances in neural information processing systems (NeurIPS) 2022 [arxiv]


Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond

Amir Feder*, Katherine A Keith*, Emaad Manzoor*, Reid Pryzant*, Dhanya Sridhar*, Zach Wood-Doughty*, Jacob Eisenstein*, Justin Grimmer*, Roi Reichart*, Margaret E Roberts*, Brandon M Stewart*, Victor Veitch*, Diyi Yang*

Transactions of the ACL (TACL), 2022 [arxiv] [reading list]


Tutorial on Causal Inference for NLP

Amir Feder*, Zhijing Jin*, Kun Zhang*

Conference on Empirical Methods in Natural Language Processing (EMNLP) 2022 [youtube] [slides (part 1), slides (part 2)]


Teaching


All Publications (*=equal contribution)


In the Eye of the Beholder: Robust Prediction with Causal User Modeling

Amir Feder, Guy Horowitz, Yoav Wald, Roi Reichart, and Nir Rosenfeld

Advances in neural information processing systems (NeurIPS) 2022 [arxiv]


CEBaB: Estimating the Causal Effects of Real-World Concepts on NLP Model Behavior

Eldar David Abraham*, Karel D'Oosterlinck*, Amir Feder*, Yair Ori Gat*, Atticus Geiger*, Christopher Potts*, Roi Reichart*, Zhengxuan Wu*

Advances in neural information processing systems (NeurIPS) 2022 [arxiv] [data]

Building a Clinically-Focused Problem List From Medical Notes
Amir Feder, Itay Laish, Shashank Agarwal, Uri Lerner, Avel Atias, Cathy Cheung, Peter Clardy, Alon Peled-Cohen, Rachana Fellinger, Hengrui Liu, Lan Huong Nguyen, Birju Patel, Natan Potikha, Amir Taubenfeld, Liwen Xu, Seung Doo Yang, Ayelet Benjamini, Avinatan Hassidim

Proceedings of the 13th International Workshop on Health Text Mining and Information Analysis (LOUHI), 2022 [pdf]


Section Classification in Clinical Notes with Multi-task Transformers
Fan Zhang, Itay Laish, Ayelet Benjamini, Amir Feder

Proceedings of the 13th International Workshop on Health Text Mining and Information Analysis (LOUHI), 2022 [pdf]


Useful Confidence Measures: Beyond the Max Score

Gal Yona, Amir Feder, Itay Laish

NeurIPS 2022 Workshop on Distribution Shifts: Connecting Methods and Applications [arxiv]


Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond

Amir Feder*, Katherine A Keith*, Emaad Manzoor*, Reid Pryzant*, Dhanya Sridhar*, Zach Wood-Doughty*, Jacob Eisenstein*, Justin Grimmer*, Roi Reichart*, Margaret E Roberts*, Brandon M Stewart*, Victor Veitch*, Diyi Yang*

Transactions of the ACL (TACL), 2022 [arxiv] [reading list]


DoCoGen: Domain Counterfactual Generation for Low Resource Domain Adaptation

Nitay Calderon, Eyal Ben-David, Amir Feder, Roi Reichart

ACL 2022 [arxiv] [code] [pdf]


Shared computational principles for language processing in humans and deep language models

Ariel Goldstein, Zaid Zada*, Eliav Buchnik*, Mariano Schain*, Amy Price*, Bobbi Aubrey*, Samuel A Nastase*, Amir Feder*, Dotan Emanuel*, Alon Cohen*, Aren Jansen, Harshvardhan Gazula, Gina Choe, Aditi Rao, Catherine Kim, Colton Casto, Lora Fanda, Werner Doyle, Daniel Friedman, Patricia Dugan, Lucia Melloni, Roi Reichart, Sasha Devore, Adeen Flinker, Liat Hasenfratz, Omer Levy, Avinatan Hassidim, Michael Brenner, Yossi Matias, Kenneth A Norman, Orrin Devinsky, Uri Hasson

Nature Neuroscience 2022 [arxiv] [pdf]


Structured Understanding of Assessment and Plans in Clinical Documentation

Doron Stupp, Ronnie Barequet, I-Ching Lee, Eyal Oren, Amir Feder, Ayelet Benjamini, Avinatan Hassidim, Yossi Matias, Eran Ofek, Alvin Rajkomar

Machine Learning for  Healthcare (ML4H) 2022 [arxiv]


On Calibration and Out-of-domain Generalization

Yoav Wald*, Amir Feder*, Daniel Greenfeld, Uri Shalit

Advances in neural information processing systems (NeurIPS) 2021  [arxiv] [pdf]


Model compression for domain adaptation through causal effect estimation

Guy Rotman*, Amir Feder*, Roi Reichart

Transactions of the ACL (TACL), 2021 [arxiv] [code] [pdf]


Learning and evaluating a differentially private pre-trained language model

Shlomo Hoory, Amir Feder, Avichai Tendler, Sofia Erell, Alon Peled-Cohen, Itay Laish, Hootan Nakhost, Uri Stemmer, Ayelet Benjamini, Avinatan Hassidim, Yossi Matias

Findings of the Association for Computational Linguistics: EMNLP 2021 [pdf]


CausaLM: Causal model explanation through counterfactual language models

Amir Feder, Nadav Oved, Uri Shalit, Roi Reichart

Computational Linguistics (CL),  2021 [arxiv] [code] [data] [pdf]


Are VQA systems RAD? measuring robustness to augmented data with focused interventions

Daniel Rosenberg, Itai Gat, Amir Feder, Roi Reichart

ACL 2021  [arxiv] [code] [pdf]


Predicting In-game Actions from Interviews of NBA Players

Amir Feder*, Nadav Oved*, Roi Reichart

Computational Linguistics (CL),  2020 [arxiv] [code] [pdf]


Active deep learning to detect demographic traits in free-form clinical notes

Amir Feder, Danny Vainstein, Roni Rosenfeld, Tzvika Hartman, Avinatan Hassidim, Yossi Matias

Journal of Biomedical Informatics (JBI), 2020 [data] [pdf]


An examination of the cryptocurrency pump-and-dump ecosystem

JT Hamrick, Farhang Rouhi, Arghya Mukherjee, Amir Feder, Neil Gandal, Tyler Moore, Marie Vasek

Information Processing & Management, 2019 [pdf]


The rise and fall of cryptocurrencies

Amir Feder, Neil Gandal, JT Hamrick, Tyler Moore, Marie Vasek

Workshop on the economics of information security 2018 [pdf]


The impact of DDoS and other security shocks on Bitcoin currency exchanges: Evidence from Mt. Gox

Amir Feder, Neil Gandal, JT Hamrick, Tyler Moore

Journal of Cybersecurity, 2018 [pdf]

Preprints


An Invariant Learning Characterization of Controlled Text Generation

Claudia Shi*, Carolina Zheng*, Keyon Vafa, Amir Feder, David Blei

Oral at NeurIPS 2022 Workshop on Robustness in Sequence Modeling [openreview]


Measuring Causal Effects of Data Statistics on Language Model's 'Factual' Predictions

Yanai Elazar, Nora Kassner, Shauli Ravfogel, Amir Feder, Abhilasha Ravichander, Marius Mosbach, Yonatan Belinkov, Hinrich Schütze, Yoav Goldberg

[arxiv]


Correspondence between the layered structure of deep language models and temporal structure of natural language processing in the human brain

Ariel Goldstein, Eric Ham, Samuel A Nastase, Zaid Zada, Avigail Dabush, Bobbi Bobbi Aubrey, Mariano Schain, Harshvardhan Gazula, Amir Feder, Werner Doyle, Sasha Devore, Patricia Dugan, Daniel Friedman, Michael Brenner, Avinatan Hassidim, Orrin Devinsky, Adeen Flinker, Omer Levy, Uri Hasson

[arxiv]


Brain embeddings with shared geometry to artificial contextual embeddings, as a code for representing language in the human brain

Ariel Goldstein, Avigail Dabush, Bobbi Aubrey, Mariano Schain, Samuel A Nastase, Zaid Zada, Eric Ham, Zhuoqiao Hong, Amir Feder, Harshvardhan Gazula, Eliav Buchnik, Werner Doyle, Sasha Devore, Patricia Dugan, Daniel Friedman, Michael Brenner, Avinatan Hassidim, Orrin Devinsky, Adeen Flinker, Uri Hasson

[arxiv]


Explaining Classifiers with Causal Concept Effect (CaCE)

Yash Goyal, Amir Feder, Uri Shalit, Been Kim

2019 [arxiv]

Community Activities


Area Chair:


Program committee member:


Workshop organizer:

First Workshop on Causal Inference & NLP (CI+NLP), EMNLP 2021 

With Katherine Keith, Emaad Manzoor, Reid Pryzant, Dhanya Sridhar, Zach Wood-Doughty,
Jacob Eisenstein, Justin Grimmer, Roi Reichart, Molly Roberts, Uri Shalit, Brandon Stewart, Victor Veitch, Diyi Yang

[url]


Tutorial organizer:

Tutorial on Causal Inference for NLP

With Kun Zhang, Zhijing Jin
Conference on Empirical Methods in Natural Language Processing (EMNLP) 2022 [youtube] [slides (part 1), slides (part 2)]