More Open

[source]

"We should pause and ask the question," says David Wiley, "is more open always better?"

The answer, of course, is "it depends". I don't keep my house open for just anyone to drop by. I keep my bank account PIN a secret. I work for the government and have a security clearance and even though we are committed to the principle of open government, we still have secrets to keep. My work on the Research Ethics Board makes clear the importance of protecting personally sensitive information. So no, more open is not always better.

That's because, as David Wiley well knows, we have other priorities in life as well. Things like social justice, prosperity for all, peace and democratic governance: these define some of our common social priorities. And we also have individual priorities: caring for ourselves and our families, maintaining good health and security, finding meaning and order. You know. It's a veritable Maslow's soup once we start looking at the ingredients of the good life.

I think everyone knows this, and so in some important ways, I think Wiley's question is a bit of a straw man. It's not the question we really need to be asking when we're talking about open, whether it's open access, open educational resources, or open pedagogy.

No, the question is this: what are these other priorities? And what overall purpose are we enlisting openness, among other things, to serve?

This can be a hard and complex question. In my own work, I define openness as one of four elements of what I call 'the semantic condition'. The other elements are autonomy, diversity, and interactivity. The semantic condition, in turn, is what informs the design of networks to enable them to learn. Networks - and therefore, people and technology and systems and the rest - that embody the semantic condition will be responsive to and adapt to the environment, and hence be resistant to cascade phenomena leading to stasis and network death. That's a lot to pack into a single paragraph, but I've written about all that elsewhere.

From this perspective, Wiley sees the world completely differently. This difference is important, since it explains most of the differences between us. Here is how he describes his priorities: 

I’m interested in developing a model that maximizes positive impacts on student success. As I’ve written before, when you’re talking about the impact educational materials can have on student outcomes, I think the key metrics are success, scale, and savings: how much can you improve outcomes, for how many students, and how much money can you save them while doing so. 

If I were forced to explain this difference, I would say that instead of adopting a design perspective, as I do, Wiley offers something more like a logic model (Wiley uses the term 'business model'), where he describes openness as one of the inputs, student success as the output, and success, scale and savings as the key performance indicators. What's important here is that each of these can be measured, and so that gives you a mechanism for balancing openness with other criteria to achieve the desired outcome.

It's not irrational. Indeed, it is far from it. It takes something really hard, looks at it from a systems perspective, and organizes it in such a way that we can define things that we can actually do in order to be successful. I've seen this sort of approach operate in companies and in governments for years.

But it's flawed in the way that these managerial approaches are flawed more generally. It depends critically on measurement, so that if you get the measurement wrong, you get the method wrong. It measures by aggregation, which means that it doesn't much matter which input we're measuring, for which person, so long as things are better overall, the method is deemed a success.

And so by using this method, Wiley can proclaim a commitment to open while doing something else. He writes:

But open is not the star of the show. It plays second fiddle to a number of things, like evidence-based learning design. (Increased access to ineffective learning materials doesn’t help anyone.) 

Here what he is doing is focusing on the metric of success. And his justification is simple. If we take the outcome of the measurement to be a product of these three criteria:

    (degree of success) x (scale) x (cost savings)

then, if the degree of success is (somehow) measured to be zero, then there is no positive outcome. So degree of success is critical to his calculation.

So what, then, counts as 'degree of success'? Wiley has explained this earlier:

Student success is the first and most important component of the framework. In the S3 framework, “success” means “completing a course with a final grade that allows the course to count toward graduation.”

I have to say, when I read this back in the fall, my reaction was: OMFG.

I mean, imagine having as a desirable outcome of a broad-based social movement something that applies only to people who have paid tuition, and have had the time and opportunity, to attend what are in the main high-priced educational institutions! Even in a country like Canada, where something like 60% of the population attends post-secondary education, most learning for most people happens completely outside that context.

From where I sit, this definition of 'success' is essentially useless for any description of open learning, because most learning that people do in their lives depends on a completely different definition of success, and even more, across a population, or even for an individual, there is no single definition of success. People define for themselves what counts as success.

Indeed, I just read and commented on a paper in OLDaily that looks at positive learning experiences in MOOCs, based on what students actually say, which even though it is based on aggregate responses, has a much broader definition of success, and one (interestingly) that doesn't talk about grades or graduation at all.

But you can see how this was such an effective argument against Chuck Severance. He, on the one hand, wants to achieve "the 'pinnacle' of open", and has done so in his Python for Everybody course, and yet in the same post, admits that:

despite the thousands of hours of time and effort he has invested to both (1) make the full chain of tools and content open and (2) show hundreds of people how to use these tools and content themselves, literally no one has done so. 

That reduces to a factor of success=0, which means that, on Wiley's model, if we take the score as a product of values, then Severance's work was literally useless. It may be 100% open, says Wiley, but "if no one takes advantage of the DIY version, how much effort does it really merit?"

Let's examine that claim. What Severance actually said is, "I have shown 100’s of people my 100% open process - and literally no one has replicated it because it is easier to just fall into the easier path of proprietary approaches." That is not the same as taking the course, the measurement Wiley uses when he asserts "as of March 1, 2021, 985,081 people have enrolled in Python for Everybody on Coursera and remember no one has stood up the tool chain themselves."

Can we say, based on this, that no one has taken advantage of the DIY version? That would be an unbelievable poor conclusion to draw. If we actually look at Severance's GitHub repository, we can see that it has been forked 1.2K times and is currently being 'watched' by 206 people. The repository has 56 contributors. And that's just the web site and course textbook. Looking at his other repositories, I see crossover from other projects, including Sakai.

I might add that nobody has replicated the Coursera course either, because Coursera is a proprietary platform, and you're not allowed to simply replicate it and host your own version of a Coursera course. So if even one person replicated Severance's value chain then the ratio, rather than being 985,081:1 would be 1:0, in other words, infinity. I'm tempted to do it, just to make a mash of Wiley's math, but I rather suspect it has already been done by people who just haven't told anyone about it.

Again, the managerial approach depends on what you measure, and if you get this wrong, you get your conclusions badly wrong (in this case, wrong by a million infinities). 

But of course all this stands aside from the real question posed by Wiley:

Now, sincerely, pause and ask yourself – which provides these students and their faculty with more access – an openly licensed GitHub repo or an easy-to-use offering on Coursera? Even when the Coursera option costs $49 – which provides more access to more people? Really think about it. In terms of raw numbers, how many people who might be interested in learning Python can come up with $49? How many people who might be interested in learning Python can get the 100% open tool chain up and running?

That's actually a pretty interesting question. And again, if you're taking the managerial approach, and using the product of success times scale times cost then you end up with a pretty one-sided conclusion.

But Wiley elides the real issue here: people don't need to get the 100% open tool chain up and running in order to benefit from the free version of the course (or the materials and such individually). They only need to get enough of it up and running so they can benefit from the free materials without cost. They don't need to set up their own instance of Sakai, for example - that would be absurd! If someone else has covered some of the cost, or if someone else has stood up the server and the tools necessary, a person can breeze in, take the course, pay nothing and benefit just as much as the person who paid $49.

Why is this important? Because it points to another of Wiley's key performance indicators, the 'reduction in cost' parameter. Because we should be asking right away, cost to whom? If you just bundle together costs to, say, an institution with cost to person, call that (say) the 'total cost of course', then the whole Coursera model looks pretty good. But wouldn't a system that produced a 'Coursera' course at a cost of zero per student be better than one that costs $49?

You could do this straight-away simply by having the government, or a foundation, or a company, or a well-endowed university pay $49 to Coursera for each person who enrolls. Coursera would earn $48.2 million for offering the course, which is what it makes from the enrollments right now, and people would have access for free!

We can all anticipate the response. If the course were free, many more people would sign up. How many? I'm not sure, and I don't have statistics from free online courses (other than my own, which - let's admit it - are nowhere near as popular) but based on everything we've seen in the world of MOOCs it would probably be in the millions. So whomever funded the course would be pretty quickly bankrupted.

But if we're just paying Coursera directly, we've eliminated a lot of the overhead of collecting money from people. And it doesn't cost $48 million to actually offer the course online. So why not fund the course directly, as before, but cap the funding at, say, $20 million, but insist that the course be offered to students for free. Surely $20 million is enough to cover the costs, even the cost of assessing assignments and awarding certificates (since it's done by computer anyway). So if even the same number of people took the course, funding it this way is twice as good, per Wiley's calculations.

So why don't we do it that way?

Well now we reach the core of Severance's remarks, the core that was deftly avoided by Wiley with his S3 framework and his metrics and his measurements: the people who fund the courses don't want to do it that way. 

Severance writes,

no one supports or invests effort into improving the free and open source production chain because (again) everyone is so busy (or lazy). This helps keep the no-free alternatives easier to use... philanthropists like Dell, Zuckerberg, and Gates and governments pour money into semi-proprietary activities because those activities feel “comfortable”. Giving money to open source hippies - not so much. The short sightedness of these well-intentioned funders makes me see red - they prop up weak companies with overly large staffs using the not-100% open models and never give a single cent to real, open models... (instead,) efforts that start open source / open content that reach some escape velocity are quickly moved away from the truly open world and towards a partially-commercial, profitable, and sustainable.

And he totally has a point. How much money do foundations pour into models that promote commercial models of open educational resources, models that very often depend on a user-pay model at some point, instead of toward models that depend on social-pay support for open access learning that everybody can use, not just those willing and able to pay the fee.

It's as though UNESCO ran it's food aid program by funding McDonalds to open new restaurants, and then subsidized the production of the bread and burgers, and called all of this 'open food access'. What people need is, first, free food to prevent starvation (that's why in rich countries we have soup kitchens even though there are plenty of McDonalds about), and then next, a way to produce their own food instead of being forced to buy it from the restaurant.

And I'm quite sure Severance would say that the reason it's so hard and takes so much effort to generate and access the totally 100% free and open course is precisely because all the funding that could be directed toward making it easier and more accessible is instead redirected toward these commercial initiatives. They come into governments and foundations and institutions with their metrics and business models and say "we can make it better", and they do, but they make it better not for the people who need it, but for themselves, and for the people who are already being well served.

And so - yeah - it does come down to what purpose it is you want 'open' to serve.