AI Replaces Pen-and-Paper at IIM Bangalore as Students Build Chatbots for Midterm Test

The Indian Institute of Management Bangalore (IIM Bangalore) has adopted an innovative approach to assessing students enrolled in its generative artificial intelligence course. Students have been asked to build their own AI chatbots and use them to answer 15 exam questions and not just do a traditional pen-and-paper midterm exam.

GenAI Exam | Photo Credit: https://x.com/IndianTechGuide
GenAI Exam | Photo Credit: https://x.com/IndianTechGuide

The unusual assessment format reflects how generative AI is changing not only how students learn but also how educational institutions evaluate their understanding of emerging technology. And rather than asking students to replicate information in a traditional examination setting, the approach puts a priority on practical application, problem-solving and working with AI tools.

As a student studying generative AI, building a chatbot can provide a more realistic assessment of the skills they are expected to develop. The exercise requires students to think about how an AI system should be built, how information should be presented to it, how questions should be handled and how the resulting responses could be assessed.

The decision by IIM Bangalore is particularly important because generative AI has created new challenges for traditional education systems. With tools that can create essays, solve mathematical problems, summarise information and answer examination-style questions, traditional take-home assignments and written assessments are increasingly difficult to measure for students' individual understanding.

Instead of trying to prevent students from using AI, the IIM Bangalore assessment appears to incorporate the technology directly into the examination process. The approach can help shift the focus from whether a student has used AI to how effectively the student can use and control AI to solve a defined problem.

In the reported format, students have been asked to create their own AI chatbots and use those systems to answer 15 midterm questions. That is, the assessment is not just about getting an answer from an existing chatbot. Students have to engage with the process of building and configuring an AI-powered tool.

Such an approach can test several competencies simultaneously. Students may need to understand prompt design, chatbot architecture, information retrieval, model behaviour and the limitations of generative AI. They may also have to assess whether their chatbot produces reliable and relevant responses.

One of the most important issues with generative AI is that there can be false or misleading information, or AI hallucinations. A chatbot may give an answer that looks good but it’s factually wrong. It’s up to students working on AI systems to recognize that and improve the reliability of their applications.

The examination model could therefore place more emphasis on critical thinking. A student who simply accepts the first response generated by an AI system may not demonstrate the same level of understanding as someone who checks the output, identifies weaknesses and improves the chatbot's performance.

The experiment also demonstrates how much university and business schools have changed in today’s AI-driven world. Industry players are becoming increasingly interested in candidates who can actually work well with artificial intelligence rather than just understand the technology theoretically.

Such an exposure to generative AI for management students in particular has wide application across marketing, finance, operations, consulting, product development and entrepreneurship. AI chatbots can be used for customer service, research assistance, internal knowledge management and many other business functions.

IIM Bangalore's reported approach therefore is more than a new examination format. It can also be seen as an attempt to connect academic assessment with the way AI tools are increasingly being applied in professional environments.

Traditional examinations generally measure how well students recall and apply information under controlled conditions. An AI-based practical assessment could instead look at how students approach a problem, choose tools, create a solution and assess the result.

We expect this shift to have effects on other educational institutions too. As generative AI becomes more prevalent, colleges and universities around the world are experimenting with new ways of assessing students. Some institutions are trying to minimize AI use; others are incorporating it into coursework and asking students to show responsible AI usage.

The IIM Bangalore model illustrates the second approach. Rather than treating generative AI as a threat to academic integrity, educators can potentially use it as part of the learning process.

AI-based assessments also raise important questions. Institutions need to determine how much of the work should be done independently, how chatbot performance should be evaluated and how instructors can distinguish between a student’s own reasoning and the output generated by an AI model.

There is also the issue of access. Students may have different levels of familiarity with AI tools, coding and chatbot development. So a proper assessment should be based on the intended learning outcomes, not the previous exposure to technology.

Despite these challenges, the IIM Bangalore experiment is a remarkable development in business education. It shows how rapidly generative AI is moving from being a topic discussed in classrooms to being a tool actively used in the classroom.

For the students involved, the midterm examination will have a very different feel than sitting in a classroom with a question paper and answer booklet. Rather, they need to build, test and use an AI system to answer the questions presented to them.

The experiment might also offer educators some insights into the future of examinations. As AI learns to do things better and better, memorizing information may not matter as much as asking the right questions, evaluating AI-generated information and using technology on real-world problems.

IIM Bangalore's reported chatbot-based midterm thus points to a wider transformation in education. The old examination could not disappear, but evaluations are likely to change as institutions look for ways to measure skills that are still relevant in a world that is increasingly AI-enabled.

By asking generative AI students to build their own chatbots and use them to answer 15 questions, IIM Bangalore is putting practical AI literacy at the centre of the assessment. The experiment reflects a growing understanding that students entering today’s workforce need to understand not only what artificial intelligence can do, but also how to use it responsibly, effectively and critically.