An Integrated-Skills Test Design for Non-English Major Undergraduate Students

YU Shu-ying

Every year， the freshmen come to the university. The university will organize them to take a placement test. The school of foreign studies will provide a test for them. This paper is concerned with a test set by teachers for the freshmen. The aim is to provide guidelines for setting a kind of test which will be practical and give helpful information to both teachers and students about the usual test. It is assumed that the teacher’s aim is to equip his students with a general knowledge of grammar and vocabulary and in the long run with the skills that they will need when they graduate and work in an actual situation. So， based on our background， needs analysis， reference and interview， it is decided that our test will be an integrated-skills one of both proficiency and placement test types which consists of listening， reading， and writing.

Keywords： integrated， placement， proficiency， freshmen

Background

Central University of Finance and Economics （CUFE） in Beijing， China is the most important university within the Ministries of Education and Finance. The university trains the students majored in accounting， trade， finance， investment， etc. It has a staff of well-committed teachers of English， Russian， French and Japanese. It has 20 schools or departments including the school of foreign language studies which is supposed to teach English to students from all schools within the university.

In recent years， China has seen an explosion in the demand for English， especially in some foreign units and enterprises. Many university students have high motivation to learn English. A good level of proficiency in English will help them considerably： to graduate from a university， to obtain better jobs， especially those in computers or joint ventures which have international connections； to read technical materials； and to study abroad. They see English as a useful world language and study it out of personal interest.

But English is a foreign language in the context of China. Students study English as a formal subject in school but have no immediate need for it outside of the classroom. They receive input in class. Our English language classrooms continue to be dominated by a blend of the Audio-Visual Method with most listening and a little spoken language and the traditional teacher-centered Grammar Translation Teaching Method. It is generally believed that the students who have fairly good performance in listening will probably be competent in speaking.

In our national curriculum， the first-year non-English major university students learn English as a compulsory subject and have to pass a 4-band proficiency level of national English exam. They pass it in the first or second year although they don’t have any English classes in the second year and they will take the main courses in finance and economics in the third year which means they will not spend a lot of time on the national English exam. If they fail to pass it， they cannot get a bachelor degree in this university. （CET Spoken English Test is optional for students in China.）

The initial plan is to give a placement test. This test “is designed to sort new students into teaching groups， so that they can start an English course at approximately the same level as the other students in the class. It is concerned with the student’s present standing， and so relates to general ability rather than specific points of learning. As a rule， the results are needed quickly so that teaching may begin” （Andrew Harrison， 1983）. If an individual student is not satisfied with the teaching group divided by the placement test， it is advisable for one member of staff to see the student individually， if only for two or three minutes， before the final class allocation is made.

The aim of this test is to place those students into three groups：

*Those who can continue their main course without English classes.

*Those who can continue their main course but will also have to continue English class to improve their English language proficiency level.

*Those who will have to take a full-time program in English in order to attain a 4-band proficiency level of national English exam.

Suppose we are asked to design a test of this kind for CUFE.

II. Needs Analysis

In order to find out the needs of non-English major freshmen for English proficiency levels—how much and what kind of English is needed， we have done a needs analysis. To get information for the needs analysis we have drawn up a questionnaire for a whole range of professions， businesses， workplaces and enterprises in Beijing where graduate students will be absorbed later. We have also referred to the College English Curriculum Requirements for English Instruction to Non-English Major Students issued by Steering Committee on Foreign Language Teaching of Ministry of Education （2020 version） to see what they expect and require for foreign language proficiency level of non-English major students. Through the analysis of the results of both questionnaire and reference， it was concluded non-English major students must， after the first calendar year of learning English as a foreign language， attain a 4-band proficiency level of national English exam， i.e.， language knowledge and skills， intercultural communication skills， speculative skills， and learning strategy requirements according to the overall description.

In addition， we have also communicated with the students who previously enrolled and have taken this kind of test before and the teachers who have designed the previous test and have participated in scoring such tests. From their responses we can repitch the level of the test， and adjust the proportion between the sections of the test and the grading system.

III. The Test Design

Based on our background and needs analysis， it is decided that our test will be an integrated-skills one of both proficiency and placement test types. “The aim of proficiency test is to assess the students’ ability to apply in actual situations what he has learnt. It is concerned with the students’ current standing in relation to his future needs” （Andrew Harrison， 1983）. Our test will consist of listening， reading， writing， use of English.

Section 1： Listening Comprehension （20 mins）

Part 1 Students listen to two long conversations and answer ten questions. Each question is followed by four multiple choices for which only one is correct.

Part 2： Students listen to three news reports or stories or passages and answer ten questions. Each question is followed by four multiple choices for which only one is correct.

Section 2： Vocabulary， Grammar and Use of English （30 mins）

Part 1： Grammar and Vocabulary

This consists of 20 multiple choice items in the form of one or two sentences with a blank to be filled in by 1 of 4 multiple choices.

（All grammatical structures are context-based）

Part 2： Close Test

Students read a passage with 10 regular gaps and fill in each blank from one of 4 multiple choices.

Section 3： Reading Comprehension （40 mins）

This consists of 30 multiple choice items based on 6 reading passages of different topics and different types.

（The text materials in this part include authentic texts from magazines and newspapers or other articles which relate to both general knowledge and simple specialized knowledge of finance and economics）

Section 4： Writing Composition （30 mins）

Students read a short article entitled “The Day My Mother Cried” and write a 150-word essay—a summary based on comprehension.

The grading scheme for this part is as follows. Each answer sheet will be marked by computers and checked by one teacher for each class and if there is a disagreement between two results， the second teacher of the foreign language department will mark it for the third time.

Section 1 Part 1： 10 marks （answer keys）； Part 2： 10 marks （answer keys）

Section 2 Part 1： 20 marks （answer keys）； Part 2： 10 marks （answer keys）

Section 3： 30 marks （answer keys）

Section 4： 20 marks （band descriptors）

total scores： 100 scores； total time： 120 minutes

Those who have scored above 80 （including 80） marks will not have to take English classes.

Those who have scored from 60-79 will have to attend English classes.

Those who have scored below 60 will have to attend a full time program in English in order to attain an integrated-skills one.

Test Administration

*Print sufficient test booklets and answer sheets. Prepare media equipment for the listening section.

*Check the test booklets， answer sheets and media equipment to make sure they have no errors or faulty sound reproduction.

*Provide instructions for invigilators and all the candidates including the classrooms， the starting time， the duration， the stationary that students should bring， the rules that they must obey during the examination， etc.

*The classroom should be quiet and big enough to avoid the incidents of cheating.

*Provide key answers and guidelines for scorers.

IV Justification of the Test Design

In language testing theory， a good test is the one which has the following characteristics： reliability， validity and practicality. The proposed test design will be justified in terms of these features.

1. Validity

*Construct validity：

Needs analysis helps the test to be valid. As described earlier， it is known that students are motivated to learn English. Based on the reasons why they learn English， we conclude that students should be able to listen comprehensively when they go abroad or work in joint ventures， to comprehend English materials， to write reports or assignments in English. That is why the component skills of three macro skills above provide the framework of the test.

In addition， it is thought that vocabulary and grammar are the core components of communicative competence. So， they are included in the test.

Furthermore， construct validity can be reflected through major trends of teaching methods （Audio-Visual Language Method and teacher-centered Grammar Translation Teaching Method） in which theories of learning and teaching are underlying. Therefore， the sections are focused on measuring grammatical and lexical knowledge， reading skills and communicative competence of the students， although there will be no direct oral examination for the whole school designed to test their speaking competence due to the practical constraints in terms of budget， time， and human resource， which can be taken by class teacher in class. Meanwhile， section four intends to measure students’ integrated skills from reading comprehension to writing composition. The intended writing piece （a summary report） is exactly based on the passage and most importantly on their ability to comprehend， summarize and write.

The content of the test reflects our understanding of language learning and the use of language in the context.

*Content Validity

From the elaboration above， the test relatively reflects the essential aspects of what the test is intended to measure in terms of grammatical and lexical knowledge， comprehension， and communicative competence.

The content of the test is basically selected by considering the purpose of the assessment which aims at measuring three skills and grammatical and lexical knowledge as well. There are all together four sections as referred in part three （the Test Design） and the content specification is relatively well-balanced， which can be apparently identified in terms of length and coverage of the test. Each skill has a representation in the test except for speaking.

In section two， testing of grammatical and lexical knowledge is more concerned with the correct use of language. The close test is intended to measure students’ inferencing ability， vocabulary， and grammatical knowledge in the specific context. “Studies， however， have shown that performance on close tests correlates highly with the listening， writing and speaking abilities” （Heaton， 1988）.

Section three is focused on measuring how well the students are effectively able to read and comprehend the written message. The last section is to intend to measure the integrated skills. Reading is communicatively combined with writing.

The text materials used in the test are authentic. They are some articles related with general knowledge and simple articles in finance and economics. Thus， the content is in accordance with the purpose of the test.

*Face Validity

The test looks right to both teachers and students because it tests what it is supposed to measure. In the listening section， students are tested on whether they can get the implied meaning of long conversations and news report or passages. In section two， there are sufficient items to cover the appropriate areas in grammar and vocabulary as the close test provides a vehicle for checking students’ mastery of vocabulary and grammatical items. In the reading section， students are tested if they can understand texts of different topics and text types. In relation to the writing section， the task is involved in reading skills and some abilities in writing.

The disadvantage of the test is that students will not bother to practice speaking skills because the test does not test speaking skills.

*Predicative Validity

The purpose of the test is to measure students’ English proficiency level in order to predict whether students can deal with their main course study with their mastery of English. Once the test taken， it is predicted that students are to be divided into three groups according to the test results.

The content of the test is based on the students’ language needs because they are motivated to work in computers or joint ventures， to read materials， to study abroad， etc. The students need the language skills in the content of the test for the prospective jobs.

The pedagogic consideration is that the items in each section range from easiness to complexity and the students will gradually be led in.

2. Reliability

In designing a test， reliability should be taken into account. This would be done in several ways to increase its reliability.

*MCQ

Multiple choice questions take up two-thirds of the test. This makes marking objective. The scoring will be done by an advanced IBM computer which operates electrically through the conductivity of pencil marks on a special answer sheet by an optical scanning system. Meanwhile， more items help the test to be more reliable.

*Controlled test design， administration， and clear instruction

The draft of the test is subjected to discussion among colleagues and supervision of the head to see if there is any vague or alternative answer for each question and typing mistakes in the booklets. The cheating on exam can be avoided because of the serious supervision of the invigilators. The format of the test is familiar to students and the instructions are easy to read. We also check if the test booklets and answer sheets are well-printed and instructions for all sections are explicit， the tapes or audio and video for listening section are well duplicated or produced and media equipment is in good condition.

*Rater reliability

As the writing part is subjective， it should be scored by personnel who have had training and experience in scoring. Moreover， marking should be checked by another rater and supervised by the head.

3. Practicality

*Cost and Benefit

First， speaking will not be tested due to the practical reasons. Second， the scoring computers will ensure to reduce the cost of the test in terms of scoring time， budget and human resources. The test is easy to be marked and be administered. It is also very easy to analyze the students’ results and then to divide them into groups. Third， the test is practical in terms of acceptable length and difficulty. At last， the test takes two hours but can exam the students’ abilities in three areas.

*Personnel for each exam room we have two examiners. Every year we have about 2，500 non-English major students. So， there should be roughly 50 classrooms with about 100 invigilators. CUFE has a staff of more than 2，000 teachers， we will not have any problem with personnel availability.

*Equipment Our institute is well equipped with computers， photocopiers， and each classroom is equipped with media. Those can help with listening section in the test， marking and analyzing after the exam.

V. Sample Test

（Section two， part two）

At the start of the 20th century， the new art medium of “moving pictures” ____41 unveiled， but it remained a debatable topic _____42 Walt Disney synchronized motion pictures _____43 sound， color， animated characters and fluid motor skills.

Disney became a legendary symbol of children’s entertainment， ______44 the cartoon character creation， Mickey Mouse. It wasn’t until the release of Steamboat Willie （1928）， however， ______45 of the very first animated sound films ever made， that other artists started to notice the potential that animation had in the film industry.

Steamboat Willie ______46 an immense contribution to the artistic and scientific intersection because of its vibrant combination of music， synchronized sound effects and the later addition of color.

Cinematography ______47 was still a new form of art in Disney’s time that many perceived _____48 a passing phrase， but when sound and color were added to moving pictures， the world of fantasy in film suddenly transformed into a more practical impression of ______49 life. _____50 that time， Disney’s Mickey mouse character paced the way to the animated film and television series that we watch and listen to every day.

41. a. were b. was c are d. is

42. a. after b. as c. until d. when

43. a. for b. on c. in d. with

44. a. alongside b.at the same time c. next to d. along the side of

45. a. few b. both c. two d.one

46. a. had been b. were c. was d. has been

47. a. only b. alone c. lonely d. then

48. a.as b. like c. for d. with

49. a. false b. real c. fake d. wrong

50.a. From b. Since c. In d. Over

（Answer：41b 42.c 43.d 44.a 45.d 46.c 47.b 48.a 49.b 50.b）

Conclusion

In conclusion， in order to take a placement test for the university’s freshmen， we have done a needs analysis and also an interview in order to find out the needs of non-English major freshmen for English proficiency levels—how much and what kind of English is needed. Based on the background， needs analysis， reference and interview， it is decided that our test will be an integrated-skills one of both proficiency and placement test types. An Integrated-Skills Test Design is expected to be enhanced to meet the trend of English proficiency and the mainstream of teachers and students. With continuous efforts and educational support， we hope to make a better future for English language testing with the implementation of the Integrated-Skills Test Design on English testing for freshmen in College.

References

Bachman， L. F. （1990）. Fundamental considerations in language testing. Oxford： Oxford University Press.

Baker， D. （1989）. Language testing—A critical survey and practical guide. London： Edward Arnold.

Harrison， A. （1983）. A language testing handbook. London： Macmillan Press.

Heaton， J. B. （1988）. Writing English language test. Longman Handbooks for Language Teachers. UK： Longman Group UK Limited.

Steering Committee on Foreign Language Teaching of Ministry of Education. （2020）. The College English Curriculum Requirements for English Instruction to Non-English Major Students. Beijing： Higher Education press.

Wang， S. R. （2019）. New generation of college English （improvement） comprehensive tutorial 2. Beijing： Foreign Language Teaching and Research Press.