LIG – Grenoble
– 1 Post-Doc, 18 months, starting July 2021
Information retrieval evaluation framework: towards continuous evaluation for industrial search engines
- Supervision: P. Mulhem and L. Goeuriot
- Starting date: July 2021
- Duration: 18 months
- Keywords: information retrieval, continuous evaluation, explainability
Evaluating search systems requires setting up an evaluation environment: select a paradigm, metrics, a dataset, etc. The choice of an environment is rarely motivated objectively, and the impact of its variations (choosing a dataset against another, altering one, etc.) is rarely measured. Such objectivity comes from a quantifiable understanding of the differences between datasets, documents, or test queries. In KoDicare, we generically call such differences “knowledge delta”. Evaluation of several environments, knowing their knowledge deltas, leads to measuring and qualifying “results deltas”. Online systems require continuous evaluation with a stable and meaningful environment that guarantees the reproducibility and explainability of systems results. A controlled environment quantifying both “knowledge deltas” and “result deltas” will support such continuous evaluation, and enable the provision of explanations for system engineers through the analysis of related changes in the two “deltas”. The theoretical results will be confronted to real cases defined by a French company that deploys a web search engine (Qwant). Currently, no such framework dedicated to real continuous evaluation of information retrieval systems exists, due to the numerous parameters that must be handled.
Aims of the project:
- Definition and formalization of knowledge deltas
- Creation of use cases
- Creation of a framework allowing the measure of the results delta
- Evaluation and analysis: correlations between knowledge delta and result delta
- Revision of the framework, towards continuous evaluation of search engines
The objectives of the hired PostDoc will be to apply the theoretical framework to the use cases. This evaluation framework will allow a comprehensive analysis of the steps of a single stage experiment. Analysis and meta analysis of the experiments will lead to an improvement of the framework towards continuous evaluation.
The work expected is the participation in the modeling of result deltas and to manage large scale continuous evaluation experiments (data acquisition from the industrial partner, bringing the knowledge and result deltas to scale in strong interaction with the industrial partner). It is expected for the postdoc to actively contribute to benchmarking activities.
Required skills for the PostDoc job:
- Knowledge in IR and its experimental evaluation
- Knowledge in ML for IR
- Strong development skills, preferably in Python, Java
- Strong interaction and teamwork skills
- Reporting and publishing skills
- Hosting institution: One of the major research-intensive French universities, Univ. Grenoble Alpes enjoys an international reputation in many scientific fields, as confirmed by international rankings. The dynamic ecosystem, grounded on a close interaction between research, education and companies, has earned Grenoble to be ranked as the 5th most innovative city in the world. Surrounded by mountains, the campus benefits from a natural environment and a high quality of life and work environment. With 7000 foreign students and the annual visit of more than 8000 researchers from all over the world, Univ. Grenoble Alpes is an internationally engaged university. A personalized Welcome Center for international students, PhDs and researchers facilitates your arrival and installation. Grenoble Informatics Laboratory (LIG) is one of the largest laboratories in Computer Science in France. It is structured as a Joint Research Center (French Unité Mixte de Recherche – UMR) founded by the following institutions: CNRS, Grenoble Institute of Technology (Grenoble INP), Inria Grenoble Rhône-Alpes, Grenoble Alpes University. The mission of LIG is to contribute to the development of fundamental aspects of Computer Science (models, languages, methodologies, algorithms) and address conceptual, technological, and societal challenges.
- Funding: The PostDoc is funded by the ANR-FWF KODICARE project (2020-2023), involving the University Grenoble Alpes (UGA), the Technological University of Vienna (TUW) and Qwant company.
- Gross salary: from 2300€/month to 2600€/month depending on the candidate experience.
Research Studio Data Science – Vienna
– 1 PhD, 3 years, starting June 2021
We are looking for a PhD student for a three year position in
Vienna, with enrolment at the Technical University of Vienna.
Data Scientist PhD Position – (f/m/d)
30 hours/week (3 years)
Your profile:Your profile: You have a university degree in Computer Science, Mathematics, or other natural sciences, and you meet the relevant criteria of the TU Wien doctoral programme https://informatics.tuwien.ac.at/doctoral/).
Desired experience:Knowledge on Information Retrieval, Machine Learning, Intelligent Data-Analysis, Text Analysis Programming / prototyping skills in Java and/or Python. Other programming languages are a bonus. Working with and querying databases, code sharing, issue tracking (JIRA, GIT).
Publications are of advantage
Excellent English communication skills, both written and verbal (e.g. IELTS at 7.0 or higher)
Your field of activity during the PhD:
- Working in an international research project of the Research Studio Data Science and towards obtaining a PhD from TU Wien, Faculty of Informatics.
- Independently and collaboratively designing Information Retrieval and Extraction experiments.
- Training and testing hypotheses, principles and models, including AI/ML models.
- Co/authoring research papers, reports and funding bids.
- Presenting y/our research at conferences, workshops and public events.
Your personal abilities and qualities:
- Understanding of project aims and specific tasks, reporting and communicating in English.
- Ability to proactively seek solutions to complex problems without needing to be micromanaged
- Ability to plan and prioritise own workload and forward plan
- Ability to do science systematically, with integrity and to follow good research practices
- Team player able to work to challenging targets
- Knowledge of German or willingness to learn
- Working in a Linux environment (deployment, essential scripting, configuration, logging, etc.).
Please send applications including cover letter, motivation, curriculum vitae and certificates to our Chief Researcher of the RSA FG by email via firstname.lastname@example.org. We look forward to hearing from you.
– 1 Post-Doc, xxx, starting xxx (TBD)
LIG – Grenoble
– 1 PhD on Information retrieval evaluation framework: towards continuous evaluation for industrial search engines
Supervision: P. Mulhem and L. Goeuriot
Starting date: February 2020
Duration: 36 months
Keywords: information retrieval, continuous evaluation, explainabilit