TEXT will refine the LLM-hard framework to explore text-based practices in education, business, and research. It will design and implement tools to address LLM-hard challenges in these areas
This work package refines the framework for evaluating complex challenges in the use of advanced language models and applies it to practical contexts in education, business, and research. It will develop and implement tools designed to overcome these challenges, ensuring that language models can be effectively integrated into real-world professional and academic settings.
Building on previous research into the limitations of existing language models, this work package adopts a design-based research approach. It examines how language models perform in specialized domains where success depends not only on generating text but also on aligning with the knowledge and practices of specific professional communities. The project focuses on three key areas: education, business applications, and research, addressing the ways in which language models can support, enhance, or challenge traditional workflows and cognitive tasks in these fields.
While language models have shown substantial improvements in generating human-like text, challenges remain in ensuring their outputs are contextually appropriate, accurate, and useful for domain-specific tasks. A major issue is that users often struggle to frame effective prompts or interpret model-generated content within their specialized fields. Furthermore, refining model parameters for better performance typically requires technical expertise beyond what most users possess. This work package seeks to bridge this gap by designing tailored technological solutions that assist users in optimizing language model interactions, refining prompts, and adapting outputs for their specific needs.
The project has two main objectives: first, to develop an analytical framework that evaluates the suitability of language models for specific professional and educational tasks; second, to design, implement, and test practical tools that help users apply language models effectively. By conducting empirical studies in diverse real-world settings, the work package will contribute insights into how artificial intelligence can be responsibly and effectively integrated into daily workflows. It will also refine the broader framework for assessing complex language model applications, ensuring continued progress in this field.
Directed by Arthur Hjorth
