An Example of Economic Analysis

Jenny says in her comment below,

What do teachers do [to] produce in students acquired knowledge and skills? Are [t]here rules, protocols, studies? Do all teachers follow them, as do accountants and doctors? No, there is little if any uniformity of practice. So how can I design an economic model if I can't measure or quantify the inputs or the functions that produce the product? And if we can't measure them, then how can we generalize about them at all?

Teachers provide the management that enables students to produce knowledge and skills. Just as in any industry, there can be a great deal of variation in the management practices that are applied. Some may be more successful than others, and most are probably situation-dependent. What is successful with one group of workers in one environment may not be successful with another group in another environment.

In order to carry out a quality analysis of an educational process, we will have to focus on an example involving a specific product, that is, one lesson. Let us choose, for today's lesson, the introduction of the multiplication tables (facts). The students will have already done multiplication in the sense of counting up groups or rows and columns of objects. Now it is time for them to learn that we do not have to count everything all the time, because we can memorize some information that will always be true and useful to have at our "mental fingertips."

Managers make choices (decisions). Some theorists go so far as to say that is their primary function. So now the teacher, as manager, must begin to make these choices. First she must clearly define the product. What is it that the students will "know" or "do" after this lesson that they did not "know" or "do" before? How many facts should be introduced in the first lesson? What level of mastery should be expected? What will be expected of students who are "ahead" or "behind?" (The fact that the textbook authors have already determined some of these things will be ignored in this discussion. The teacher may not make all of these decisions on a daily basis, but still has the responsibility to determine what is appropriate, that is, whether the textbook should be followed or alternative lessons used.) Obviously we are talking about goals and behavioral objectives in educational terminology, and product specifications in industrial terminology.

Second, the teacher must design a production process, or in educational terms, a lesson plan. Should worksheets be used? flashcards? a computer program? Should the students work in groups and quiz each other? Should there be a contest, or should it be a non-competitive activity? Note that the teacher is planning how the students will WORK! (Remember the question, "Who are the workers?") Also, note that I have given a list of possible "rules or protocols" that are well established in the industry. Indeed, it is likely that there are numerous studies of the effectiveness of these techniques available in the education literature, as well as in the multitude of unpublished master's and doctoral theses in math education. To suggest that there are not well-established procedures or techniques available in education is absurd, and we are churning out research on the available alternatives at a rate unprecedented in educational history. Having alternative production procedures available should not be considered a bad thing. Uniformity in industrial practice might suggest, in some cases, that the best possible practice has been found, but it might also suggest a lack of incentive to improve. Indeed, improvement is impossible if uniformity is enforced. Furthermore, lack of uniformity in practice may be a reflection of varying conditions under which production occurs. That is to say, students are different, schools are different, and teachers are different. It makes sense that the process would vary as well.

Third, the process must be implemented. The workers must be informed of their functions, given motivation or incentives to accomplish the task, and the tools, materials, and environment to carry out the work. Again, these are management functions.

Fourth, Quality Control (or Improvement) must be carried out. In education, we call this assessment. Assessment occurs on two levels, and they have clear analogies in industry. One level is the assessment of individual students, or grading. This corresponds to product inspection. Management decisions must be made on how to handle defects. In other words, what should be done with students who did not meet the objectives? The other level is institutional assessment, which is really the goal of quality improvement. We want to evaluate the process, perhaps generating descriptive statistics of the performance of the group and compare them with statistics from other groups who used a different process. If we don't have numbers, we rely on the teacher's judgment of the results. Is she satisfied, or will she try something different next time? Unquestionably, this is the point where we want to apply statistical analysis and yet, will find we have great difficulty in the educational setting. In industry, the comparisons are done on data acquired from randomized experiments, so that hypothesis tests are a reasonable basis for a decision. Using data from non-randomized trials is extremely problematic, due to confounding variables and the inability to determine cause-and-effect relationships.

As for the question about "if I can't measure or quantify the inputs or the functions that produce the product," I would have to say that difficult does not equal impossible. Indeed, we can measure inputs and other aspects of the process. There has been a great deal of progress in the field of cognitive psychology recently, where carefully designed experiments are conducted, and the theory that develops can be applied to processes carried out in the classroom. Unfortunately, education theorists do not always take notice of the results from this field.

Economic Analysis

The business/management analogy I discussed previously could be considered a sort of economic analysis. Economics doesn't just mean money. The idea is that people engage in goal-directed activities in order to realize a gain (profit) of some kind. Even altruistic activities can be seen this way, if intrinsic rewards are also considered profitable.

Really, this is nothing more than a way of analyzing goals and the processes required to achieve them.

Many of our institutions are structured so that the activities carried out in them are expected to result in a profit. This is even true of non-profit organizations, because the term "non-profit" really means that the profit isn't in the form of money, but some other desired outcome, such as alleviated suffering, happier communities, etc. Government institutions are structured to profit society as a whole in various ways. Schools are no different.

We would like to improve our schools. We want to make instruction better and more efficient. By defining the activity in economic or business terms, we can have some hope of achieving these goals. It is not that business terminology is necessary to do this. But the quality control literature has provided a framework for improvement that has had phenomenal success in the last forty years or so. Some people would object that reducing an activity as personal as education to the cold, dry terms of economics is unreasonable or even immoral. The idea that economic analysis is cold and dry, though, is not necessarily accurate. Nor does such analysis, properly done, eliminate the human element of the process.

Let's look at another trend in education. The use of "research" and "statistical analysis" has become predominant in the education literature in any discussion of improving education. Yet, in the past, people objected that such scientific analyses were inappropriate for education. This kind of research is limited in its effectiveness because of the large number of confounding variables present in any study. It is also rare that randomized experiments can be carried out in education, therefore results can be biased and statistical methods appropriate only to randomized experiments are often used inappropriately in the analysis.

Scientific research and economic analysis are by no means mutually exclusive. They are complementary. Economic analysis provides a conceptual framework in which to implement and interpret research.

More on the management analogy

I have been carrying on a discussion with JennyD in the comments section of my May 5 article, Are Teachers Responsible for Student Learning? She didn't like the industrial analogy to education very much. However, in answering her comments, I have been more convinced that the analogy is valid. My article should be read carefully, and with thoughtfulness. Don't be trapped by "jargon" and misinterpret or misrepresent what is written. No analogy is perfect, and this one isn't meant to be either.

Saying the teacher has the role of manager does not mean that all teachers do is "classroom management," a term that has a rather narrow meaning in educational parlance. Managers have roles in planning, motivating, promoting, selection of resources, production targets, and many other things.

I have said that education is the only business where the customer is the product. That's not exactly right, but it does cloud the issues for this analogy. It also explains part of why education is a "messy" business. In quality control (Demming, Taguchi, etc.) the production process is broken down into small steps or stages. The team at each stage needs to analyze its part in the process, defining who its customers and suppliers are, what the customer requirements are, and what parts of the process can be changed to reduce variation in the product, as well as to streamline the process. The customers and suppliers can be internal or external to the company, and are found at various levels. Suppose you work in a factory attaching wheels to lawn mowers. Your immediate customers are the workers at the next station down the line. Your intermediate customers might be retailers, and of course the ultimate customers are the people who buy the lawn mowers to use them. The quality of your work can be assessed by inspection and customer satisfaction surveys.

Suppose now you are a 3rd grade teacher doing a lesson in math. The students are customers who have certain requirements (as expressed on their behalf by parents and school board). The product they request is a mathematical concept or computational skill. Now I ask you to consider: What is it you have been hired to do? Can you produce the concept or skill and transfer it to the customer? Or are you a middleman, transferring the knowledge or skill from some warehouse that others have stored it up in? Certainly not. Only the students, by their own mental effort, can construct a concept, or practice a skill until it is automated. The teacher cannot do this work for them. Therefore, the students are the production workers. They make the product (learning) happen. What then, is the role of the teacher? The teacher provides an environment in which the work is to be done. The teacher analyzes the product requirements and plans a process that will produce the product. The teacher guides the students to do the necessary work, and is responsible to motivate them and evaluate the quality of the result. These are all MANAGEMENT functions.

I do not propose this as some kind of new fad for modeling instruction. It is just one way to understand some aspects of what happens in education. You can use other analogies to explain other aspects. I said earlier that this model explains why education is a "messy" business. Several business functions are muddled. The customer is the product. The workers are not the ones being paid to get results. The raw materials are highly variable, input quality cannot be controlled, and quality of the product is difficult to measure, in part because the product is never really finished.

Summer Schedule

I will try to post occasionally this summer but I will be at a location with only dial-up access and that makes all internet work painfully slow. Web sites just aren't designed for dial-up any more.

Are Teachers Responsible for Student Learning?

JennyD asks the question, linked in the title. There are many good comments on her article. Please visit her site and read them. I want to relay this story from a colleague of mine, which I think is very relevant.

Dr. L teaches Quality Control. The students are mostly in Mechanical Engineering. When he talks about analyzing a production process, he asks the students what role they take, and what role he takes, in the production process. The roles to consider are Manager, Worker, Customer, and Product. He says that invariably, they say that he is the worker and they are the customers. This leads to a lively discussion.

OK, if you are the customers, what is the product you are buying? Knowledge.
So, if I am the worker, then I create knowledge in you? Yes.
Do I really have that ability? Don't you think that if I could actually create knowledge in you I wouldn't be here, I'd be on the lecture circuit making megabucks selling my secret? Well....
So who really creates the product? We do.
That makes you... The workers.
And then I must be... The manager.

I'm not sure if he answers the question of who the customers are. In a sense they could be the students, the parents, the future employers, or society. All of these have a role in paying for the product, and all benefit from it.

But my main point here is that the students are almost stunned to think of the teacher as the manager rather than the worker. They have been led to believe, all through their educational experience, that the teacher is their employee, and is responsible for providing them with a product. This example shows that is not quite the case. The role of the manager is to organize work in a productive, efficient process that will result in a quality product. The teacher does not produce the product himself, he organizes the work of the students so that they will produce it. If he is a good teacher and has a good workforce, the product will have high quality. Otherwise...

See also the Instructivist's article on this subject.

[Apologies--I didn't realize I was referencing articles a couple of months old. Still, it was a good excuse to post this story.]

More Moore

On the second day, Ted Mahavier talked about sources for Moore Method Course Materials. There is much available on the web. There was a report on a project called Uteach, and another on a study comparing Montessori and Moore methods (Montessori has expanded into middle and high school). Apparently they have much in common.

Ed Burger, of Williams College gave tips on teaching Moore-style.
1. Everybody submits written (typed) proofs for each problem. If he has two sections, he has the other section anonymously referee.
2. Curb your comments!
3. Allow presentation of only one proof and discuss it. The only questions are, is it clear? correct? complete? Students will want to show alternatives, but that is restricted to their written versions.
4. Always look dubious. Question everything. Don't give inadvertent clues about whether it is right or wrong.
5. Never look at the speaker. Scan the class instead.
6. Use index cards for each student to assess. Keep track of passes, record proofs presented, with little notes of anything you want to remember when grading.
7. Most problems are given with instructions to "prove and extend or disprove and salvage."
8. Assess failure--have them write up an essay on how they learned from their failures.

In the afternoon there were K-12 sessions. Bill Jacob presented on "Using Video of Children Engaged in Inquiry in Courses for Pre-Service Teachers." I had to leave early to catch my flight after that.