Valid Reliability & Reliable Validity

Student Achievement Concepts:

1. Reliability Comes by Default of Validity

     LeBlanc (2009,) discusses an assessment may be reliable, but this does not mean it is valid, a valid assessment though does guarantee reliability. This concept is important to be mindful of when establishing the validity and reliability of an assessment and an action plan for student achievement. If enough reflection and review of the validity of an assessment/plan is taken into account, prior/post administration of the assessment/plan, then the level of reliability will be appropriate and sufficient. Educators need to have the flexibility and comprehension to ensure student achievement is occurring measuring up to a higher level of validity.

2. High Validity

     LeBlanc (2009), also discussed the validity of high-stakes tests and their validity having a correlational coefficient of at least 0.8 (a high positive correlation). LeBlanc is implying a high level of validity must be established by these tests to ensure student achievement. Listed below are questions an educator must ask themselves and/or others concerning high validity for student achievement.

The questions a reflecting teacher should expect/review when determining high validity for an assessment:

1. Does this particular reading test meet the instructional objectives reflected on by the assessing teacher?
2. Does the reading content presented by the teacher reflect upon the content presented to the students?
3. Has the appropriate cognitive reading level been established by the assessing teacher and reflected upon?
4. What will be the proper scoring criteria (rubric, etc.) used for this reading test?

The questions a reflecting teacher should expect/review from their students' responses during and after the given assessment:

1. What observations will the teacher make during the assessment period?
2. (Reflection) After the assessment period, what observational changes need to be made during the assessment?
3. What questions should a teacher ask student(s) (interview) post-assessment time, for future consideration and reflection?
4. What are the potential consequences prior, during, and after the assessment, pluses and deltas?

3. Measurement of Error

     Kizlik (2011), mentions we measure to obtain information about what is, and such information may or may not be useful, depending on the accuracy of the instruments we use, and our skill at using them. It is crucial to remember there is always a measurement in error regardless what is being measured (quantitative and/or qualitative gathered data). So even these established rules and standards we measure by, there maintains an error of measure. Educators need to remember, error is underestimated (McMilan 2000). To maximize a high level of student achievement we, as educators, must remember to minimize the measurement of error for our students through our assessments and action plans.


Teacher & Administrators Concepts:

1. Assessment is inherently a process of professional judgment.

     Whether that judgment occurs in constructing test questions, scoring essays, creating rubrics, grading participation, combining scores, or interpreting standardized test scores, the essence of the process is making professional interpretations and decisions (McMilan 2000). We as educators, in our differing judgment(s), need to remember being objective is relative to the defining individual. Understanding this principle helps teachers and administrators realize the importance of their own judgments and those of others in evaluating the quality of assessment and the meaning of the results (McMilan 2000). The essence of wisdom is the action educators take to ensure a high quality of achievement for students and themselves professionally.

2. Assessment decision-making is influenced by a series of tensions.

      Competing purposes, uses, and pressures result in tension for teachers and administrators as they make assessment-related decisions (McMilan 2000). Educators are consistently under pressure concerning critical assessment-related decisions. I assume, each school is different in their approach to the decisions made and the results discovered. We must remember, that we will try to accommodate all the tensions that exist, but in the end results to optimize our educational environments, trade-offs are inevitable (McMilan 2000).

3. Assessment influences student motivation and learning.

     What is the nature of feedback, and when is it given to students (McMilan 2000)? According to LeBlanc (2009), student feedback should always be accomplished and presented immediately-to soon after the attempted assessment. How does assessment affect student effort (McMilan 2000)? In my observations, if the expectations were high initially prior to the student(s) attempting the assessment, then post-attempt, those students who felt they under achieved persevere (regardless of debate) as expected. Answers to such questions help teachers and administrators understand that assessment has powerful effects on motivation and learning (McMilan 2000). McMilan (2000), states for example, recent research summarized by Black & Wiliam (1998) shows that student self-assessment skills, learned and applied as part of formative assessment, enhances student achievement.

References

Kizlik, B. (2011). Measurement, Assessment, and Evaluation in Education. ADPRIMA. Retrieved March 1, 2011 from http://www.adprima.com/measurement.htm

LeBlanc, P. (2009). Key Ideas in Validity and Reliability for Teachers. Southeastern University. Retrieved March 1, 2011 from http://www.youtube.com/watch?v=IF-oeuidRuU

McMilan, J. H. (2000). Fundamental Assessment Principles for Teachers and School Administrators. Practical Assessment, Research & Evaluation, 7(8). Retrieved March 1, 2011 from http://PAREonline.net/getvn.asp?v=7&n=8

Research with Benefits

As a student working on my internship project, the one benefit that currently stands out to me, is an appreciation for the time and effort taken, by individuals and/or groups, to discover and/or add-on to the research that hopefully benefits all it pertains too. The time spent on reflecting on what to research, how to research it, where to start, when it should be completed by or if ever, why is it important, who will it affect/not affect, and how does one research to give coherence to an educated audience, is awesome task and accomplishment in my estimates.

I can truely apprecite the segments from the videos discussing how to 'fill the gap' or 'the missing puzzle piece'. I admire the cognitive learning that takes place when performing the research, from the cyclic reflections to the fomalizing of coherent transitional thoughts and ideas brought into summary. Dr. Classen, from NC St.,  mentioned in the second video that in developing our own ideas while conducting the research, we are picking up a story and adding on to it, or as I would note, telling a story with a enligtened factual twist. Again our research will fill in the gap(s) or become a solved puzzle piece to the larger inter-webbed picture of a topic.

I like to know that I am a critical-independent thinker and will be looking for the relevance/difference/agreement/disagreement between the sources of information I use to explore. I will find what is critical and significant to the research and what is unecessary. I will see the agreements and disagreements between th esources, and what source aligns to others and who it opposes. In all fairness, being new at this process, I will try and do my best to decifer truth from fiction and separate the flaws from fact, so that it may not inhibit my research process and analysis.

Round Robin

Quan vs. Qual vs. Action Research

Video 2: Horowitz

There is one line that was mentioned by Horowitz that I believe gets to the point concerning research, "...questions drive the method, not the method driving the questions." When I was in elementary school, we had an half hour a day to read whatever we wanted. I chose to read the choose your own adventure Dundgeon and Dragons books. Why? because my choices led to a newly discovered outcome every time I went on these literary journeys. Our questions, concerning research should do the same, lead to new discoveries with the questions we derive. Our questions will lead to new acquired information only if we use them properly to drive the method.

Videos 4 & 5: Dr. A.G. Picciano

Dr. Picciano did an excellent job describing the different aspects of quantitative and qualitative data. I learned something new about each of the methods that will help guide me concerning my projects to come.
As far as quantitative research, I was familiar with the mathematical related concepts and the descriptive/correlational/experimental/survey research, but Ex Post Facto research (newly learned) will definitely be implemented in my future projects. As far as qualitative data, essence (who, what, why, where, when, how) was mentioned. Of course I have heard of these one-word questions before, but to describe them all in one word helps simplify the research process. The essence of the research should drive the method.

Video 6: Participatory Action Research (Military)

This explanation of his dissertation is highly helpful in understanding the direction of the action research process. His points from the visual concerning creating a adaptive curriculum, will help me comprehend the goals to achieve form my research. It is interesting it will take cycles (as mentioned in video 7) to acquire the results. PAR is a tremendous path of direction to follow to understand change and and potential improvement of educational practices. The action research results (1st-3rd persons) will be used in my research practices as well. I did not realize before this video information, I was already using these practices in prior surveys and questionnaires I have used in the past. The section discussing stories and data is useful in relating the results in a more appetizing informative format. Data becomes stories and I can analyze the data for stories.