This paper describes design and implementation of multiplication table learning support system with spoken dialogue. Our system provides multi-modal interface for learning. It has speech synthesis engine for speaking to a child (learner) and speech recognition engine to recognize learner's voice responses. Multiple sensitive organs are used in learning, looking at the display (by eyes), listening the voice of the system and their own (by ears) and speaking (by mouth). Our system is designed in investigating learning activities in an elementary school and implemented in accordance with the progress (phase of comprehension) of each study. We define the following three modes according to learning phases. They are a memorize mode, a practice mode and a test mode. To achieve quick recognition and high recognition rate, we constrain vocabulary and syntax the system accepts, and we also extend vocabulary to recognize multiplication table specific numeration.