Teacher Assessment Literacy:How do We Know What We Need to Improve?

Chris Davison

Introduction

Teacher assessment literacy is regarded as one of the most influential factors in improving student learning in the classroom， in particular a teachers ability to collect， interpret and use a range of assessment information to help students monitor and evaluate their learning needs， set achievable goals， and use targeted feedback from teachers and peers to improve their learning （Black & Wiliam， 1998； Hattie， 2008）. This paper will first unpack some of the key concepts underpinning teacher assessment literacy and the development of an assessment for learning culture， building on the authors work in China， Singapore and Brunei. The paper will then focus on a case study of one Australian collaborative approach to building teacher assessment literacy， the Tools to Enhance Assessment Literacy for Teachers of English as an Additional Language （TEAL） project， which is designed to help teachers of students with English as a Second or Additional Language （ESL/EAL） to use assessment tools and techniques more effectively so as to improve teaching and learning. The tools include four main components： first， a set of sequenced teacher professional learning resources about English language learners and assessment designed for small group or self-directed study； secondly， an assessment tool bank containing a range of assessment tools and tasks， including computer-adaptive tests， organized around the three broad macro-skills （oral， reading and writing）， three macro-functions（informative， persuasive， imaginative）， three stages of schooling （early elementary， mid to upper elementary， and lower secondary） and a range of English language proficiency levels； thirdly， a range of assessment-for-learning and teaching exemplars including a selection of annotated units of work across a range of subject areas and year levels showing assessment tasks with formative feedback embedded within a teaching/learning cycle， and finally， an online teacher discussion forum， including a password-protected area for teachers to share problems and strategies and to moderate work samples in order to build a community of assessment practice. The paper discusses the rationale for the selection of the resources for teacher assessment literacy in English language education and their potential to make a difference to teachers and students. The implications in terms of the process of defining and describing teacher assessment literacy for other systems and settings will also be discussed.

What is teacher assessment literacy （and why is it important）？

Teacher assessment literacy is regarded as one of the most influential factors in improving student learning（Black & Wiliam， 1998； Hattie， 2008）， however there is no common definition of assessment literacy among researchers and policy-makers. Some see it as data literacy， in particular， the ability to interpret results of standardized tests； others talk about it as the possession of knowledge about sound assessment principles and practices， for example， as in New Zealand schools（Ministry of Education， n.d.）：

“Assessment literacy is the possession of knowledge about the basic principles of sound assessment practice， including its terminology， the development and use of assessment methodologies and techniques， and familiarity with standards of quality in assessment.”

Why is assessment literacy seen as the “new” Holy Grail， or as Popham（2009） ironically describes it， a bona fide “magic bullet” for education？ Much research evidence suggests assessment is the key to improvement in learning. Black and Wiliam （1998） have convincingly demonstrated the learning gains that can be achieved through well focused teacher-based formative assessment enhance students learning more than any other strategy across age levels and in different contexts. Hattie （2008） in a study of major influences on educational achievement （using more than 800 metaanalyses） found that formative practice， in particular self-assessment and feedback， had the highest effect sizes （i.e.， impact on student outcomes） out of more than 100 different instructional and contextual factors. For these reasons worldwide there has been a push to improve teachersassessment literacy， as well as that of other stakeholders， in part due to concerns that teachers may lack sufficient training in what educational assessment entails and/or may lack the confidence or skills required to perform their assessment duties competently. As Popham （2009） says， “…assessmentliterate teachers will typically make better decisions … because we want students to be better taught. It should be obvious that todays teachers must acquire more assessment literacy.”

This raises the question of what kind of “assessment” we want teachers to be literate in？ Perhaps not surprisingly， given the traditional and high-stakes role of assessment in educational systems， researchers do not agree on how assessment literacy should be defined nor what it might comprise. Although many have attempted to describe this construct（De Luca & Klinger， 2010； Malone， 2013； Mertler， 2009； Plake， 1993； Popham， 2009， 2011； Siegal & Wissehr， 2011； Volante & Fazio， 2007； Xu & Brown， 2016）， Stiggins （1991） convincingly argues that the meaning of assessment literacy varies due to “the needs of the decision maker and the practical realities of the decision context”. This implies that each educational system needs to develop its own definitions and descriptions of teacher assessment literacy to suit its own particular assessment context， as there is so much variation in what constitutes assessment knowledge and skills （for example， see Black & Wiliam， 2005； Davison 2004， 2007）. Only then will we know what needs to improve.

This is the case for the Australian school context， in which teacher-based assessment accounts for most evaluation that takes place. In terms of its assessment demands and practices， Australia is very different from other contexts， as it tends to sit in the middle when comparisons are made between it and other countries， both in terms of its performance on international assessments； and in terms of its assessment system which is balanced between classroom level assessment and standardised assessment， with a strong focus on classroom assessment， and a long history of prioritizing assessment for learning over assessment of learning. In assessment for learning cultures， research suggests that teachers spend from onequarter to one-third of their professional time on assessment-related activities， so assessment literacy needs to focus more on a teachers ability to collect， interpret and use a range of assessment information to monitor and evaluate learning needs， provide targeted feedback and help students set achievable goals than in interpreting and using the results of standardized tests， though that is still important.

For example， in the largest and most diverse state educational system in Australia， New South Wales （NSW） （NSW Education Standards Authority， n.d.）， the syllabuses advocate assessment for learning：

“…this is a type of quality assessment that has had world wide success in enhancing teaching and improving student learning. Assessment for learning gives students opportunities to produce work that leads to development of their knowledge， understanding and skills. Teachers decide how and when to assess student achievement， as they plan the work students will do， using a range of appropriate assessment strategies including self-assessment and peer assessment.”

Hence， assessment for learning emphasizes the interactions between learning and manageable assessment strategies that promote learning， clearly expresses for the student and teacher the goals of the learning activity， and reflects a view of learning in which assessment helps students learn better， not just achieve a better mark. It provides ways for students to use feedback from assessment， helps students take responsibility for their own learning， and is inclusive of all learners.

Research shows that NSW teachers are more likely to use formative assessment （including feedback to students） than the average of OECD teachers and teachers are encouraged to use a variety of assessment techniques that are valid， reliable and appropriate to the age and stage of learning， although Australia still has some way to go to ensure that teachers understand how to interpret and understand assessment data and effectively embed assessment within teaching and learning. A 2013 Staff in Australias Schools survey reported that 25.7% of primary teachers identified the need for more professional learning in making effective use of student assessment information. The findings were similar for secondary teachers. In Australia， concerns have been raised about the trustworthiness and reliability of teacher assessment decision-making processes and teachersability to be both “accurate” and “fair”； and also about teachers capacity to be able to collect and use appropriate information to improve learning. However， they are not mutually exclusive—both are important for effective assessment， with the teacher equally， if not more， accountable to individual learners， not just systems， and systems highly motivated to improve learning， not just rank learners.

Te a c h e r a s s e s s m e n t literacy in English language education

Paradoxically， the concept of teacher assessment literacy has only relatively recently been widely discussed and promoted in the English language education field （for example， see Davison， 2017， 2019； Inbar-Lourie， 2008； Lam， 2015； Scarino， 2013； Taylor， 2009； Tsagari， 2011； Tsagari & Vogt， 2017； Xu， 2019）. This is partly because of the traditional dominance in the field of large-scale standardised externally set and assessed tests， but also because assessing ESL/EAL learners is a particularly challenging area for most teachers.

In Australia， the primary focus in English language education until relatively recently has been on developing more accurate， consistent and transparent descriptions of EAL development to improve reporting systems， especially definitions of the target groups for funding purposes. Much less attention has been paid to improving teacher assessment literacy at all levels， despite the growing emphasis on benchmarking student performance against standardised assessment outcomes which has created particular difficulties for teachers working with learners from diverse linguistic and cultural backgrounds generally， especially those teachers with a variable level of language knowledge and assessment literacy. In Australian schools， about 30% of students require systematic and regular English language support， hence increasingly all teachers—not just EAL teachers—need access to appropriate and useful assessment tools and advice to enhance assessment literacy in order to support learning and teaching.

E n h a n c i n g t e a c h e r assessment literacy： One approach to improving teacher knowledge and skills in Australia

To respond to this need， researchers at the School of Education at the University of New South Wales， in partnership with the Victorian education system and Educational Assessment Australia（EAA）， developed an innovative online assessment “tool-kit” to help all teachers develop pedagogically sound approaches to assessing the English language development of all students. Drawing on Assessment for Learning （AfL） principles and Vygotskian theory， the Tools to Enhance Assessment Literacy for Teachers of English as an Additional Language（TEAL） is for use by all Victorian school teachers to help assess the stage of development for EAL students in speaking and listening， reading and writing， to improve learning and teaching. All tools are aligned against the Victorian EAL curriculum， with potential for alignment to other standards by other jurisdictions.

Following a practice-what-youpreach model （Hill et al， 2014）， TEAL was developed by adopting the same principles and promoting the same practices for teachers that needed to be implemented with students in Australian schools， that is：

by finding out initially where teachers were in terms of their assessment literacy；

sharing learning intentions， setting clear and coherent success criteria and achievable timelines， modelling desired outcomes and leading—by example—to sustainable improvements in assessment， learning and teaching；

putting the learners and teachers at the center of the change process；

involving students， parents and the wider school community in understanding and supporting reforms；

maintaining confidence in the assessment system， and striving to be theoretically and philosophically consistent.

At the beginning of the project development， over 150 experienced EAL teachers were surveyed to gather their own perceptions of their assessment literacy， using a suitably contextualised description of teacher assessment literacy developed by Alonzo （2016）， in collaboration with this author， which prioritized collaboration and student engagement， that is：

“Teacher assessment for learning literacy （comprises the） knowledge and skills in making highly contextualised， fair， consistent and trustworthy assessment decisions to inform learning and teaching to effectively support both student and teacher learning. Teachers aim to build students and other stakeholders （such as parents and school leaders） capabilities and confidence to take an active role in assessment， learning and teaching activities to enable and provide the needed support for more effective learning.”

The survey showed that the EAL teachers participated in professional development related to assessment and many were undertaking further education/ training in assessment. The majority demonstrated a belief in the ability of every student to improve， and felt confident they could identify appropriate teaching methods and tailor lessons to available resources， maintain confidentiality in dealing with assessment results， develop an environment of trust， use assessment to build students interest to learn， and reinforce the positive learning attitude of students. However， somewhat unexpectedly the same group of teachers indicated very low selfefficacy in a number of key components of assessment for learning， with a confidence level below 60% for the following very common assessment activities：

gathers a range of evidence of student learning， 2.36；

designs English language assessment tasks， 2.45；

engages students in peerassessment， 2.45；

engages students in self-assessment/ reflection， 2.68；

involves students in the development of learning outcomes， 2.87；

gives feedback related to criteria， 2.87；

assists students in using feedback to feed forward ， 2.98；

collaborates with family to establish home activities to support students， 2.98.

This survey informed the development of tailored assessment tools and advice through a process of researcher-teacher collaboration involving the active input and training of hundreds of EAL specialists from selected government， Catholic and independent schools in Victoria and over 10，000 EAL students and their families. The action-research included the collaborative collection of detailed video and documentary records of over 1500 exemplars of student language use in authentic primary and secondary classroom settings， and the trialling and validation of over 60 different oral and written assessment tasks， and associated rubrics， for the assessment tools bank， consisting of a range of assessment tools and tasks organized around macro-skills cross-referenced by assessment type， EAL stages and year levels， as well as the development， trialling and validation of an innovative computer-adaptive test for vocabulary and reading assessment.

However， in developing the online teacher-mediated ESL assessment resource center， priority was also given to the collection and provision of teacher professional learning resources to enhance teacher assessment literacy， including background material on assessment for learning principles and processes， and video and text-based resources to showcase various aspects of assessment literacy. In addition， the site included a selection of annotated units of work across a range of subject areas and year levels， showing assessment tasks with self and peer assessment and formative feedback embedded within a teaching/learning cycle， and， most importantly， an online teacher discussion forum—a passwordprotected area for teachers to share problems， strategies and work samples and engage in moderation/benchmarking. Nearly five years later， this has resulted in a sustainable teacher-based assessment system aligned with the EAL curriculum to provide longitudinal data and reports to all key stakeholders on students English language and literacy development over time， but at the same time innovative and dynamic scaffolding for the continuing development of teacher assessment literacy. The underpinning Vygotskian framework of teacher assessment literacy as activity system and scaffold is described by Michell and Davison （2019） .

In the final stages of the project implementation， a pilot professional learning program was developed to evaluate and enhance the capacity of schools to use the TEAL website to improve their learning and teaching of EAL students， and assess and report on their progress. It also functioned as a field trial to refine and improve the TEAL website， and to track any changes in teacher assessment literacy over time and identify areas requiring improvement. From February to November 2016 there were six rounds of professional learning with 10 groups of three teachers from each school （EALD， content-area， leader）， funded by the Department of Education and Training （DET） Victoria， and taught by the University of New South Wales（UNSW）， consisting of 6 x 3-hour or 3 x 6-hour modules， focusing on：

becoming familiar with TEAL， its rationale and aims， in particular， developing an understanding of the philosophy of assessment for learning and giving appropriate feedback to students；

developing a clear understanding of the materials and assessment advice on the website；

reflecting on how to incorporate TEAL materials into the assessment of EAL learners；

having opportunities to give feedback on the TEAL website.

All sessions incorporated time for professional dialogue and for sharing of strategies to enable school-based implementation， plus action-oriented activities in the form of between-session tasks， such as trialling and evaluating TEAL common oral and written assessment tasks， experimenting with feedback or self and peer assessment， as well as readings and online reflections. Overall， a total of 182 teachers enrolled in the professional learning programs—a mix of primary and secondary government， Catholic and Independent schools， along with some teachers from P-12 Colleges and intensive English language centers.

Teachers were asked to complete a pre/post program survey which showed that over the course of the program all teachers gained a greater understanding of TEAL as well as feeling more ready and confident to implement it in their schools. Comparison of pre/post program feedback showed a doubling of participants understanding of the TEAL website， assessment for learning principles and their use in their classroom as well as significant improvements in their confidence in improving teaching and learning for EAL learners in their school and their understanding of EAL learner needs as well as their knowledge and skills in providing EAL students with feedback and understanding of how to implement student peer and self reflection with EAL students.

In the post program survey， teachers identified the most useful aspects of the TEAL resources， including the actual assessment tasks， the criteria sheets， assessment samples， advice on feedback and self and peer assessment， the readings and resources， the links to the EAL Curriculum and the sample unit planning formats. Participants also provided feedback on the professional learning program， indicating that they gained most from：

1. Getting to know and use the TEAL website：

Spending time exploring the resource and implementing it into my program. If wed merely been told about it， it would still be sitting untouched in a folder somewhere.

As a school leader， TEAL is useful for leading other staff and delivering general EAL advice and PD. The site has some great resources for this.

Great opportunity to try out a range of assessment tasks， criteria sheets， readings and resources.

2. Networking with colleagues and sharing ideas and expertise，eg：

Good to have all this professional learning with a large group of EAL teachers and learning from the presenters； Time to talk with others， share ideas and make connections.

The time to sit with colleagues and discuss assessment issues and structures within the school.

Ability to take this information back to inform whole-school planning going forward and to provide PD for teachers.

3. Acquiring knowledge about assessment for learning and how it empowers EAL learners and maximizes their learning， eg：

A wake up call about assessment for learning and not just data collection. I can do much better！

Promoting success for those who need it most.

Strategies and awareness of assistance to EAL students will benefit the entire cohort of students by catering for all needs.

The program has inspired me and helped me recognise the need to develop a whole school vision and implement less formal yet consistent and varied forms of assessment for our EAL learners.

Conclusion

TEAL is seen as an important resource for building teacher assessment literacy and professional learning in English language education， not just in Australia but internationally. It also provides an innovative mechanism for the process of defining and describing teacher assessment literacy and a model of what could be developed by other systems and settings wishing to develop their own unique support structures for the improvement of assessment literacy in their context.

References

Alonzo， D. 2016. Development and Application of a Teacher Assessment for Learning （AfL） Literacy Tool[D]. Sydney： UNSW.

Black， P. & Wiliam， D. 1998. Assessment and Classroom Learning[J]. Assessment in Education： Principles， Policy & Practice， 5（1）： 7-74

Black， P. & Wiliam， D. 2005. Lessons from Around the World： How Policies， Politics and Cultures Constrain and Afford Assessment Practices[J]. Curriculum Journal， 16（2）： 249-261

Brookhart， S.M. 2011. Educational Assessment Knowledge and Skills for Teachers[J]. Educational Measurement， Spring 30： 3–12

Davison， C. 2007. Views from the Chalkface： English Language School-Based Assessment in Hong Kong[J]. Language Assessment Quarterly， 4（1）： 37-68.

Davison， C. 2013. Innovation in Assessment： Common Misconceptions and Problems[M]// Hyland K. & Wong LL. （eds）. Innovation and Change in English Language Education. London： Routledge.

Davison， C. 2017. Enhancing Teacher Assessment Literacy in English Language Education： Problems and Pitfalls[R]. In Plenary Presented at the Applied Linguistics Conference （ALANZ/ALAA/ALTAANZ）. Auckland.

Davison， C. 2019. Using Assessment to Enhance Learning in English Language Education[M]// Gao， X （Ed.）. International Handbook of English Language Teaching， Vol. 1， Second edition. Dordrecht： Springer.

Davison， C. & Michell， M. 2014. EAL Assessment： What do Australian Teachers Want？[J]. TESOL in Context， 24（2）： 51–72

De Luca， C. & Klinger， D. A. 2010. Assessment Literacy Development： Identifying Gaps in Teacher Candidates Learning[J]. Assessment in Education： Principles， Policy & Practice， 17（4）： 419–438

Engelsen， K.S. & Smith， K. 2014. Assessment Literacy[M].// Wyatt-Smith， C.， Klenowski， V. & Colbert， P. （eds）. Designing Assessment for Quality Learning. The Enabling Power of Assessment. New York： Springer.

Hattie， J. 2008. Visible learning： A Synthesis of over 800 Meta-Analysis Relating to Achievement[M]. London： Routledge.

Hermansen， H. 2014. Recontextualising Assessment Resources for Use in Local Settings： Opening up the Black Box of TeachersKnowledge Work[J]. Curriculum Journal， 25（4）： 470–494

Hill， M. F.， Ell， F.， Grudnoff， L.， et al. 2014. Practise What You Preach： Initial Teacher Education Students Learning About Assessment[J]. Assessment Matters， 7， 90–112

Inbar-Lourie， O. 2008. Constructing a Language Assessment Knowledge Base： A Focus on Language Assessment Courses[J]. Language Testing， 25 （4）： 385–402

Lam， R. 2015. Language Assessment Training in Hong Kong： Implications for Language Assessment Literacy[J]. Language Testing， 32（2）： 169–197

Malone， M. E. 2013. The Essentials of Assessment Literacy： Contrasts Between Testers and Users[J]. Language Testing， 30 （4）： 329–344

Mertler， C. A. 2009. Teachers Assessment Knowledge and Their Perceptions of the Impact of Classroom Assessment Professional Development[J]. Improving Schools， 12（2）： 101–113

Michell， M. & Davison， C. 2019. Bringing the Teacher Back in： Toward L2 Assessment Praxis in English as an Additional Language Education[M].// Poehner， M. & InbarLourie， O. Toward a Reconceptualization of L2 Classroom Assessment： Praxis and Researcher-Teacher Partnership[M]. New York： Springer.

Ministry of Education New Zealand， n.d. Assessment Literacy[Z/OL]. [2019-04-01] . https：// assessment.tki.org.nz/Assessment-forlearning/Assessment-for-learning-inpractice/Assessment-literacy

New South Wales （NSW） Education Standards Authority， n.d. Assessment for Learning in Years 7-10[S/OL]. [2019-04-01]. https：// arc.nesa.nsw.edu.au/go/7-8/assessment-forlearning-in-years-7-10/

Plake， B. S. 1993. Teacher Assessment Literacy： Teachers Competencies in the Educational Assessment of Students[J]. Mid-Western Educ Res 6（1）： 21–27

Popham， W. J. 2009. Assessment Literacy for Teachers： Faddish or Fundamental[J]. Theory and Practice， 48（1）： 4– 11

Popham， W. J. 2011. Assessment Literacy Overlooked： A Teacher Educators Confession[J]. Teacher Education， 46（4）： 265–273

Scarino， A. 2013. Language Assessment Literacy as Self-Awareness： Understanding the Role of Interpretation in Assessment and in Teacher Learning[J]. Language Testing， 3（4）： 309–327

Siegel， M.A. & Wissehr， C. 2011. Preparing for the Plunge： Preservice Teachers Assessment Literacy[J]. Science Teacher Education. 22： 371–391

Stiggins， R. 1991. Assessment Literacy[J]. Phi Delta Kappan， 72 （7）： 534-539

Stiggins， R. 2014. Improve Assessment Literacy Outside of Schools Too[J]. Phi Delta Kappan， 96 （2）： 67-72

Taylor， L. 2009. Developing Assessment Literacy[J]. Annual Review of Applied Linguistics 29： 21–36

Tools to Enhance Assessment Literacy （TEAL）， n. d. For Teachers of English as an Additional Language[Z/OL]. [2019-04-01]. http：//teal. global2.vic.edu.au/

Tsagari， D. 2011. Investigating the“Assessment Literacy” of EFL State School Teachers in Greece[M]// Tsagari， D. & Csépes， I. （Eds）. Classroom-based Language Assessment. Peter Lang， Frankfurt am Main.

Tsagari， D. & Vogt， K. 2017. Assessment Literacy of Foreign Language Teachers Around Europe： Research， Challenges and Future Prospects[J]. Papers in Language Testing and Assessment， 6（1）： 41–63

Volante， L. & Fazio， X. 2007. Exploring Teacher Candidates Assessment Literacy： Implications for Teacher Education Reform and Professional Development[J]. Canadian Journal of Education， 30（3）： 749–770

Xu， Y. 2019. English Language Teacher Assessment Literacy in Practice[M].// Gao， X （Ed.）. International Handbook of English Language Teaching， Vol. 1， Second Edition， Dordrecht： Springer.

Xu， Y， & Brown， G. T. L. 2016. Teacher Assessment Literacy in Practice： A Reconceptualization[J]. Teaching and Teacher Education， 58： 149-162

Chris Davison，澳大利亞新南威爾士大學(xué)教授，教育學(xué)院院長(zhǎng)，新南威爾士州教育委員會(huì)主席。