Friday, December 22, 2006

Comments on The Bioteaming Manifesto

Interesting paper - I don't like the Change This delivery vehicle though (text is always too small, and I can't position it (I always like to read text at the top of the screen, not the bottom).

The metaphor is a bit strained. Why 'bio' teams? I get the analogy, but aside from mentioning it from time to time they do not actually connect what they're saying with the metaphor they're using. And some parts are beyond credulity. "The four action zones are to a bioteam what the four chemical bases (A,T,G and C) are to DNA - their interdependencies and constantly repeating patterns provide the building blocks of the double helix structure common to all living things..." Um, no. The 'action zones' are sets of rules. They do not form helix structures, or any other structures.

Overall: the advice in the paper is generally sound and includes several things I haven't thought of. The whole section on beliefs, for example, is very valuable. Now, not all of the 'beliefs' listed are actually beliefs - some are attitudes, others are behaviours. Nonetheless, they describe an essential mindset.

But: Good beliefs make teams work harder? Not exactly, and not exactly necessary. Wording hard is a matter of motivation, of which the beliefs (etc) constitute partial factors. Also, working harder is not always preferred to working efficiently and effectively. There needs to be a connection drawn between factors and beliefs to effectiveness and efficiency, but this is not done.

Also, like other papers of this type (Siemens's book comes to mind) even as they say things like "everybody's a leader" they're still writing to the manager and describing to the manager how to manage. Eg. why do ghey continue to call them 'teams'? They are still working in, clinging to, the paradigm of groups and leaders while trying to assimilate network behaviour. Eg. "HPT members don;t believe their leader will take all the glory..." Not an issue if there are no leaders per se.

The Factors: 'intelligence' isn't a factor - it's not really a part of the composition of teams (or networks, as I would say). It is a consequence. The second factor, autonomy, is a factor. But where is diversity? I don't see it.

I found rule 1 interesting - "they mostly use one-way broadcast communications..." Where does this come from? Broadcasting information rather than orders makes sense, but setting up a structure based on broadcast messages is a bit less convincing - it sounds like a way to sneak orders back into the picture, albeit implicitly. If your 'team' depends on no-reply all-staff emails, then it is not a network.

Looking out for threats and perceiving opportunities - this is the behaviour of a group, not a network. And this raises an important point underlying the article as a whole - the fostering of thinking on the part of members of the whole as a whole. Basically, what entails in the personification of the whole on the part of members, and then the self-identification with the whole. This creates cohesiveness, but also introduces a set of dysfunctional behaviours - less tolerance for diversity, for example. An attitude of assimilation rather than connection.

Rule 8 - tit-for-tat - is an attempt to explicate an important organizational principle. It is one I have expressed in the based as "exchange of mutual value". The idea that relations between members are not based on coercion but rather negotiated micro-contracts.

Rule 9 is an important principle and deserves much more discussion - the distinction between analysis and planning to get thinsg right vs a process of live controlled experimentation. Other people have described this as 'tolerance for small failures'.

Tuesday, December 19, 2006

Response to My letter On AIDS

A month or so ago, as part of the global campaign on AIDS, I sent letters to Brian Murphy, MP, and Tony Clement, Minister of Health. My letter was a customized version of the template sent out by the Make Poverty History, underlining my personal commitment to this issue. Brian Murphy responded promptly with printed letter; this was received today from the Ministry.

December 15, 2006

Mr. Stephen Downes
[stephen@downes.ca]

Dear Mr. Downes:

The Honourable Josée Verner, Minister of International Cooperation and Minister for La Francophonie and Official Languages, received a copy of your message to the Minister of Health, regarding HIV/AIDS. She has asked that I respond to you on her behalf.

Canada is playing a leadership role in ensuring a comprehensive and integrated response to HIV/AIDS, particularly in developing countries. The Canadian International Development Agency (CIDA) works with partners at all levels, including Canadian civil society, the international community, and developing country partners. Over the past five years, CIDA has provided approximately $600 million in funding to support the implementation of initiatives, such as prevention programs and engaging youth, to effectively respond to the HIV/AIDS epidemic in the developing world. In 2006, the new Government of Canada announced an additional $250 million over two years for the Global Fund to Fight AIDS, Tuberculosis and Malaria, of which approximately 60 per cent will go towards HIV/AIDS.

In addition, on World AIDS Day, December 1, 2006, this Government outlined its long-term, comprehensive approach to fighting HIV/AIDS globally and announced $120 million for the first in a series of new and concrete initiatives to combat the disease. These investments form part of the Government's commitment to improving peoples' health in the world's poorest countries and build on Canada's new aid effectiveness agenda.

Canada recognizes that many developing countries will need assistance in implementing the World Trade Organization’s decision of August 30, 2003, which allows developed countries to export generic versions of patented drugs to developing countries unable to manufacture their own. We are working with our partners to assist developing countries to take advantage of Canada’s Bill C-9, including putting in place mechanisms to procure, disburse, manage and prevent diversion of pharmaceutical products. The Government of Canada continues to work with the pharmaceutical industry, civil society and developing country partners to ensure that the medicines that are needed to address public health problems such as HIV/AIDS are available and affordable to those who need them. Minister Clement has announced that he is launching a review of this legislation in response to criticism that the legislation is too complicated.

Canada remains a strong supporter of the Heavily Indebted Poor Countries (HIPCs) debt relief initiative. Through the Canadian Debt Initiative, HIPCs completing the process receive 100 per cent forgiveness of their debts owed to Canada. For example, Canada recently announced the cancellation of all debt owed to Canada by Cameroon as this country reached completion point in the HIPC process. Canada has also contributed to multilateral trust funds that help international financial institutions to provide their share of HIPC debt relief without negatively impacting their financial status.

Canada and other G7 members recognized that more needed to be done to reduce HIPCs’ debts to multilateral financial institutions such as the World Bank and the International Monetary Fund (IMF). Thus, donors agreed to the Multilateral Debt Relief Initiative (MDRI) in the fall of 2005. The MDRI provides for 100 per cent cancellation of HIPCs’ debts owed to the IMF, the International Development Association (the concessional lending facility of the World Bank) and the African Development Bank, once they complete the HIPC debt relief process. Canada is providing its share of the costs associated with this new initiative. In turn, Canada hopes that these countries will make good use of the fiscal space provided by these two debt relief initiatives to invest in their growth and poverty reduction.

Regarding Canada's level of development assistance, consistent with Canadians’ compassion for the less fortunate, the government will advance Canadian values and interests on the international stage by providing much-needed assistance to the world’s poor, as part of our commitment to meet the Millennium Development Goals. Budget 2006 reaffirms the government’s commitment to double international assistance from 2001–02 levels by 2010–11. In line with this commitment, Canada’s international
assistance will grow to about $3.8 billion in 2006–07 and then to approximately $4.1 billion in 2007–08. We are also committed to exploring how to move Canada towards the average ODA/GNI ratio, as set out by the Organisation for Economic Co-operation and Development.

The government remains committed to continuing to increase, within a prudent fiscal framework, its international assistance over the short and long term. More broadly, this government is committed to supporting Canada's core values of freedom, democracy, the rule of law and human rights around the world. Canada is providing families and communities with the means to lift themselves out of poverty and build a better life.

For further information on recent and ongoing initiatives, I invite you to visit the CIDA Web site at www.acdi-cida.gc.ca.

Thank you for your interest in this important issue.

Yours sincerely,
Stephen Wallace
Vice-President
Policy Branch

Platform for the Revolution

In a two part article, The Revolution Misses You (Part One, Part Two) Zack Exley argues that we need an economic revolution - but states that people have to work in order to make it happen. "But right now, without econ PhDs, without industrial leaders, without scientist sages, we’ll just have to get this process started ourselves. Come back for the next post and we will."

Exley makes it sound as though nobody has thought about the problems and that nobody has proposed solutions. This “I asked a girl and she didn’t know” dodge is as old as the hills. I would expect that young supporters of capitalism aren’t well versed in economic theory either.

But in any case, numerous solutions and policy points have been advocated. I might add that thousands (or more) of economists, scientists and even business people are working across a broad front of NGOs, public agencies, and as private citizens, in support of this new agenda. The press may treat them as though they don’t exist. But the proposals are nonetheless made.

Let’s look at the WTO talks specifically. On the ground, the response you would most often get if you were not cherry-picking your interviewees is that world trade liberalization must go hand in hand with would standards on environment and labour. It helps nobody if people in democracies are required to compete with what essentially amounts to slave labour managed by dictators.

A more sophisticated approach - and the issue that caused the collapse of the WTO talks in Mexico - was the issue of subsidies. To over simplify, the rich nations want to eliminate subsidies in areas where they are strong, such as manufacturing or cultural industries, and to keep them where developing nations have a chance to compete, such as in agriculture. The developing nations want the principle of subsidies applied equally. And who can blame them?

More recently we have seen the activism of the Live 8 concerts and the associated movement designed to eliminate poverty. One of the major reforms advanced by this organization focuses around debt relief. This is especially important for developing nations. After their U.S. or Soviet sponsored dictators absconded with the state treasury, the poor nations were left holding the bag. Debt relief and debt elimination would go a long way toward reversing their fortunes.

Another matter has been raised regarding the hold speculators have over money markets; this was indeed the primary cause of the collapse referenced in the article. Global free trade has always countenanced the unrestricted mobility of global capital, but this has allowed speculators to hold nations hostage (and, of course, global free trade has never contemplated a corresponding free movement of people, who must continue to be held hostage behind nationalist fences). People working against this have proposed reforms such as those adopted by Chile, while slow down this global movement of money.

Advocates have also directed their attention toward the IMF and the World Bank. These agencies, paying homage to capitalist principles, have insisted that recipient countries cut social programs dramatically in order to qualify for debt servicing support. opponents have argued that the development of a social infrastructure, including some level of income support, is necessary in order to give these countries the stability they need to survive and to lower the corruption that hinders economic activity.

Countries that have opposed these conditions have been treated as pariahs by world economic markets and their client governments and media. Consider how nations such as Cuba, Venezuela and Bolivia are treated by the media. Consider how even simple measures, such as the adoption of open source by governments in Brazil and Peru have been countered by commercial software companies and their governments.

Indeed, commercial intervention into foreign national economies through force of arms and other disruptive tactics is only now coming to the fore. A film like Blood Diamond comes years after the Kimberly declaration. But the fostering of insurrection and armed conflict over other resources continues. Activists do not forget the killing of Ken Saro Wiwa in Nigeria over oil leases, even if the media does. And of course capitalist intervention in the middle east occupies everyone’s minds these days.

Of course this imply points back to a respect for rights and freedoms. The unfettered exercise of capitalism is essentially a license for corporations to disregard any of the hard-won victories for the people over the years. From simple things, such as Wal-Mart’s union-bashing activities in Quebec, Canada, to the labour camps described in Klein’s No Logo and elsewhere, the corporate agenda has been harmful wherever it has landed.

The right to freedom also characterizes many of the economic agendas that have operated in the online community. There is a large and expressive open source movements that has railed against such things as software and business method patents as economically ruinous. An additional community lobbies extensively for open access to educational and research materials, reasoning that this work, funded by taxes levied on the people, properly belongs to the people.

Activists are also opposing the privatization of community knowledge, privatization in the form of genetic and other life patents, commercialization of cultural property and other artifacts, the withholding of life-saving AIDS and other drugs, and in general, the committing to the private market the combined public property of the world. This ongoing privatization is an enormous transfer of wealth that benefits only those who are already wealthy, while furthering the impoverishment of those who are poor.

On a more local front, anyone who looks will find no end to solutions being proposed. In Canada, for example, we have the Canadian Centre for Policy Alternatives, an economic think-tank devoted to progressive and humanitarian economic policy (of course, our media cites the radical right wing Fraser Institute on an almost daily basis, while completely ignoring the much more qualified CCAP).

This organization, and others like it, advocate for measures that protect people from economic disaster. In Canada, for example, the CCAP (along with numerous other organizations) supports our piublic health care program that protects families from losing their homes and their livlihoods due to illness. Additional income support programs, such as unemployment insurance and welfare, form a substantial safety net (it is easy to criticize such programs - easy, that is, until visiting the slums of nations that do not have them).

In a similar light, advocates for public education continue to prevail in some nations, opposing the ruination of the system that is being fostered by capitalist privateers promoting a cornucopia of charter schools, vouchers, management firm and outright sales. It is incredible that the education of the next generation would be left in the hands of people whose bottom line is the profits to be gained, but this is the norm in the developing world and increasingly the norm elsewhere.

To say that no solutions have been proposed is a blatant falsehood. To say, even, that the common anti-WTO demonstrator has no grasp of these issues is probably also misrepresentative. The preceeding has been only a short summary of the alternative approach for a just world economic order. The detailed programs exist, and millions of people are working toward them worldwide, hindered by those very economists who scoff at the egalitarian dreams of a little girl.

Monday, December 18, 2006

Full vs Partial Feeds

Responding to Dan Lockton:

Not all feeds are designed to spread the article-length content of blogs.

One of my own feeds, for example, will at first glance resemble a partial feed.

On closer examination, my feed turns out to be linking to longer posts from other sites. This sort of blog - sometimes called a link blog or a summary blog - is actually older in pedigree than the more modern blog, and in fact more resembles the original use of RSS - to provide content for what are now called webtops.

You couldn't fit a whole article on a webtop, and that would really defeat the purpose. The whole idea of a summary (as in 'Rich Site Summary') is to provide enough info to make a choice.

Now of course there are commercial sites that use summaries simply to drive traffic. They also use links to drive traffic - that doesn't make links bad. What needs to be examined is not the practice but the intent. If the intent is to provide a certain type of service - as mine is - then that should validate the technique.

Indeed, while I've pretty much reconciled myself to the current state of blogging, it has always bothered me that the RSS produced by blogging engines - every last one of them - produces 'link' elements that point back to the original site, instead of to some third party topic of discussion.

This virtually by itself ensured that blog publishing would be primarily egoist, which links to other sites something that would take extra effort and additional encoding, making it much less prevalent than would have otherwise been the case.

Imagine what the web would have looked had the default in blog posts and RSS feeds been as it was originally used, to point to someone else's site. Imagine the conversations and linkages that would have produced.

Instead, the current system focuses on browsing RSS by source rather than by interest, with sites like my own the exception rather than the rule, and so the merging of content and ideas that could have happened has progressed at a much slower pace than it would have otherwise.

Sunday, December 17, 2006

Frustrated With Drupal

OK, I admit it, I am very frustrated with Drupal right now.

I have spent days trying to do something that should be very simple - to set up Drupal with a system that allows me to send out my email newsletters.

It didn't help that my web host was in the middle of switching mailing list managers, from ezmlm to mailman. It didn't help when my ste was set up for mailman, but documented for exmlm.

After finally threatening to move my account elsewhere they actually sent me a terse four-line message telling me where and how to set up a mailman mailing list (the lists are all on another server so if you don't know where it is, you're stuck. And CSoft uses its own controlpanel, csoftadmin, which has its own unique command set).

In the interin, I tried a bunch of things with Drupal. I got to explore, for example, the Drupal Actions and Workflow modules. There is no documentation available for actions - not even something that sys you need to use actions in conjunction with some other manual. Nothing. And with workflow, the only documentation I could find was a video.

Drupal really really has to fix this. The search results thrown up by the site on any search are utterly useless. You get bits and pieces of discussion in the Drupal discussion forums - questions about how to install it, for example, with no links to the answer. What you don't get (maybe because it doesn't exist?) is documentation on how to use it.

Anyhow, after spending several hours figuring our how to make the workflow work, and to send an email message, I run into another problem: how to send an HTML message. It appears to have become very political inside the Drupal developer community, with the verdict being that the makers of Actions will only allow the purity of text email. Morons!

More burrowing through the Drupal site. Try the Send Module, which in turn requires the MimeMail Module. According to the site, Send is supposed to "permits you to create a 'send this item' link on any node type (image, story, event, etc.)." What it doesn't tell you is how you are supposed to do this. If there is a trigger or a toggle anywhere inside the module commands, I couldn't find it (and I spend hours searching, methodically going through every possible setting - I am nothing if not persistent (people who read these articles should know that by now)). So, another dead end. I cannot use these modules to send anything.

So anyhow. CSoft finally gets around to telling me about the new mailman installation. OK, fine. Forget trying to use Drupal to send emails, I'll just do that manually, and use the Drupal Mailman Manager module to allow people to subscribe and unsubscribe from mailing lists. I set up the lists in mailman (again working without proper documentation, so I'm working by feel here). I am able to subscribe and unsubscribe using my administrator user ID. Then I test on a typical website user. Disaster.

  • warning: array_merge_recursive() [function.array-merge-recursive]: Argument #1 is not an array in /home/downes/www/drupal/modules/user.module on line 2139.
  • warning: uasort() [function.uasort]: The argument should be an array in /home/downes/www/drupal/modules/user.module on line 2142.
(The links in the error message are, of course, are useless, pointing to nothing.)

I poked around in the code a bit - for some reason, the user.module code is more than 100K, with entire help files hard-coded in it. The problem is here:

    function _user_forms(&$edit, $account, $category, $hook = 'form') {
    $groups = array();
    foreach (module_list() as $module) {
    if ($data = module_invoke($module, 'user', $hook, $edit, $account, $category)) {
    $groups = array_merge_recursive($data, $groups); [2139]
    }
    }
    uasort($groups, '_user_sort'); [2142]
    return empty($groups) ? FALSE : $groups;
    }

So what we have happening here is that the output from one function fails as input to the next function. There is, of course, no type checking or anything like what quality code would perform; it just breaks.

I tried a couple things to fix it, but I am simply not familiar enough with this software to fix it.

Just adding to my frustration is that something - I know not what - is bogging the site down. Could be my service provider, which sometimes hangs for no apparent reason. Could be Drupal, which is by now very bloated (I will totally have to reinstall once I know which modules work and which ones don't).

Now I'm just stymied. The search on Drupal for help on mailman is, of course, useless. There's another mailman module, but t would require that I write my own server. There's an ezmlm module, but of course that doesn't work.

So how can I send nice HTML newsletters to my subscribers? I have no idea right now. And I've started searching - for other service providers, for other content management systems. I've had about enough of this.

I know that Drupal is in the middle of moving to a (not yet functioning) Drupal 5 system. But it has to:

- enforce quality control on modules, at least, those that appear on the Drupal site
- create documentation for modules, real documentation, that tells people how to use the module
- fix the search function so that you aren't flooded with useless discussion list messages

That's a start. I would actually redesign the module pages themselves. The Support section, which appears on every module page, for example, links to general Drupal forums and to bug reports and submissions. This is not support.

I don't know what to do now. I'm way part my deadline for moving the site. I have been fighting with Drupal for so long now I just don't know if I have the will any more. I hate PHP; it really is spaghetti-code. Can I simply set up my new server to use my own Perl code (which, at least, works)? I don't know - the people behind Perl created perl modules, which are a huge barrier for any implementation.

I'm not sure what I want to do now.



Wednesday, December 13, 2006

Alive in Baghdad

A few days ago, the link to the site Alive in Baghdad was posted on ITForum. When the site first appeared as a candidate for the Vloggie awards in early November, my impression was pretty negative. Maybe I was wrong; who knows? Not the point. The point is that ITForum erupted, first with a rant about terrorists and a very negative denouncement of the site, and then with a series of posts saying the matter was off-topic and should be kept off the site. In all fairness, it's unlikely any of the members of ITForum were aware that this was a prominent site being supported for an award. Still, not the point.

To me, the point is this (and this is what I posted to ITForum): "The point of the post is to show how Iraqis are using internet technologies and informal learning techniques to educate themselves and the rest of the world about what is really happening in Iraq. The whole point is that the internet liberates, empowers." And a day later, I wrote, "from my perspective, all the stuff about training and competencies and the like are off-topic, distorting the landscape with a very overt corporate and conservative agenda, an e-learning as job-training agenda. But when you live in a world where learning has everything to do with work and earnings, and nothing to do with freedom and empowerment, the training and earnings stuff is on-topic and the freedom stuff is a needless distraction... I don't expect you to agree with me - but I do expect you to recognize that what is neutral and benign to you is often very biased and politically loaded to me."

Today the moderator of the list responded with an interesting and informative set of posts. First, she linked to Katy Campbell, Richard A. Schwier and Richard F. Kenny, who write, "designers have not necessarily recognised their agency in the development of a knowledge economy that reflects culturally biased views of teaching, learning, and the construction of knowledge." She also linked to a lecture by Clare Brant on the topic of Mi'kmaq Ethics and Principles, interesting to me because his document reflects very closely my own attitudes and beliefs.

And I will say that I spent enough time in the Canadian north and among First Nations peoples to know that they (and Canadians in general) are not above making a pointed remark when the time is appropriate. And maybe the time is appropriate. Which is why I appreciated this post from Mike Klonsky very directly connecting support for the war with various educational reform efforts discused here and on the list. And to be perfectly blunt: "The War In Iraq Costs $347,004,942,917. Instead, we could have provided 16,822,035 students four-year scholarships at public universities."

I think that's clear enough. And though it may not win me any awards, I will not surrender the moral high ground to people who believe they can justify the bombing of a city over the educating of a child. And to quote the other thing I said on ITForum: "this sort of nonsense in Iraq has gone on long enough, and it is the failure to speak up that has let it happen." It's time, don't you think? The right time, and yes, the right place.

Tuesday, December 12, 2006

Elite Institutions and Qualified Students

Responding to The Failure of Critical Thinking by John V. Lombardi

It is disappointing and more than a little disturbing to see a professor write with such little regard for his opposition but also for the principles of reason he lauds so misleadingly in his article.

To his opposition he should at least show the courtesy of accurate representation. The deemed emphasis on elite universities is entirely imagined, and moreover, it is not argued that elite universities should admit every needy undergraduate that exists.

Indeed, he seems very misinformed on just what it is that is supposed to make these institutions elite. In this article, it seems that they are elite because they charge more tuition. However, what is intended to make them elite is that they have the highest standards and admit the very best students

This - and not the straw man excuse offered by the author - is why elite universities should offer scholarships and other supports for needy students. Even though they have had to cope with disadvantages, students who are very needy can be as qualified as students who are very rich.

The possibility that universities might be concerned about the academic capabilities of their new students, and that they might operate using some sort of merit-based screening, does not appear to occur to the author. This lapse is startling. Is he not aware of the SATs and other evaluation mechanisms? Is he not aware of the admissions process at his own institution?

Instead, we are treated to the ridiculous parody of an argument, to the effect that "The elite status of an institution comes from its ability to spend more money than institutions deemed 'non-elite'"” and that the "amenities define elite status for undergraduates."

Even were the poor subsidized, argues the author, "that doesn’t really work" because if the poor students are admitted, then some qualified students from higher income groups would be pushed out. "There are not enough spots in what we call elite institutions to accommodate all the deserving students of all income levels."

One wonders where he dug up this ridiculous bit of logic. Were selection performed based solely on merit, then the number of qualified students equals the number of spaces, since if there are x number of spaces, then to 'qualify' is by definition to be in the top x number of students.

The author's definition of 'qualified' appears to be somewhat different, however. Exactly what it is, he doesn't tell us, but it appears to be some selection process that excludes poor people, so he can make up the fantasy of evicting qualified higher income students were those poor students admitted.

While it is true that for each poor student admitted a richer student much instead attend some other institution, it is a fallacy to say that this richer student was qualified. Only if you suppose that academic ability plays no part in the selection - the absurd premise underlying the entire article - does the statement become true.

The fact is, when elite institutions preferentially admit the rich, they fail not only society but themselves as well. To spend additional money providing preferential treatment to people who are already wealthy is a poor use of community resources. And it robs the institutions of students worthy of their status.

Perhaps the author should spend some time reflecting on the purpose of academic institutions, rather than wasting his effort defending the rich from imagined slights and unfair treatment from an ungrateful society.

Friday, December 08, 2006

A New Website, Part Thirteen - The Conversion

I've completed the script transferring the data from my old site to the new one. Once I get the mailing lists set up I will be technically ready to switch over (haven't heard any updates from CSoft yet).

The contents mostly copy over OK - all of the data ends up in the database in the right place, which is good. No additional discoveries about the database to report; I just needed to get the conversion script to work.

So, without further ado, here's the script. My data converts to Drupal. Now I need to figure out how to view it (there being utterly no instruction on how to actually view the newly created content types). The dearth of discoverable Drupal documentation continues to amaze me.



#!/usr/bin/perl

$|++; # Turn off print buffering
use DBI;



print "Content-type: text/html\n\n";
print "Test";

my $dbh = &db_open("DBI:mysql:xxxxxx:localhost","xxxxxx","xxxxxx") or print "Database connect error: $!";


print "Database is open";
my $rec_num = 0;



print "Preparing Node records:";

print "Articles: ";
my $sth = $dbh->prepare("SELECT * FROM post");
$sth->execute();
while (my $ref = $sth -> fetchrow_hashref()) {

next unless ($ref->{post_type} eq "article");
unless ($ref->{post_crdate}) { $ref->{post_crdate} = time; }

$node->{nid} = $rec_num;
$node->{vid} = $rec_num;
$node->{type} = "content_article";
$node->{title} = $ref->{post_title};
$node->{uid} = 1;
$node->{status} = 1;
$node->{created} = $ref->{post_crdate};
$node->{changed} = $ref->{post_crdate};
$node->{promote} = 1;
$node->{moderate} = 0;
$node->{sticky} = 0;
$vars->{idfield} = &db_insert($dbh,"node",$node);

$node_content_article->{vid} = $rec_num;
$node_content_article->{nid} = $rec_num;
$node_content_article->{field_body_value} = $ref->{post_content};
$vars->{idfield} = &db_insert($dbh,"node_content_article",$node_content_article);

$node_revisions->{vid} = $rec_num;
$node_revisions->{nid} = $rec_num;
$node_revisions->{uid} = 1;
$node_revisions->{title} = $ref->{post_title};
$node_revisions->{timestamp} = $ref->{post_crdate};
$node_revisions->{format} = 0;
$vars->{idfield} = &db_insert($dbh,"node_revisions",$node_revisions);
#print "Inserting record number $rec_num. Title: ",$ref->{post_title},"
";
exit unless ($node_revisions->{title});
$postcodes->{$ref->{post_id}} = $rec_num;

print ". ";

$rec_num++;
}

print "Done Articles";


print "Authors: ";
my $sth = $dbh->prepare("SELECT * FROM author");
$sth->execute();
while (my $ref = $sth -> fetchrow_hashref()) {

unless ($ref->{author_crdate}) { $ref->{author_crdate} = time; }

$node->{nid} = $rec_num;
$node->{vid} = $rec_num;
$node->{type} = "content_author";
$node->{title} = $ref->{author_name};
$node->{uid} = 1;
$node->{status} = 1;
$node->{created} = $ref->{author_crdate};
$node->{changed} = $ref->{author_crdate};
$node->{promote} = 1;
$node->{moderate} = 0;
$node->{sticky} = 0;
$vars->{idfield} = &db_insert($dbh,"node",$node);

$node_content_author->{vid} = $rec_num;
$node_content_author->{nid} = $rec_num;
$vars->{idfield} = &db_insert($dbh,"node_content_author",$node_content_author);

$node_revisions->{vid} = $rec_num;
$node_revisions->{nid} = $rec_num;
$node_revisions->{uid} = 1;
$node_revisions->{title} = $ref->{author_name};
$node_revisions->{timestamp} = $ref->{author_crdate};
$node_revisions->{format} = 0;
$vars->{idfield} = &db_insert($dbh,"node_revisions",$node_revisions);

$authorcodes->{$ref->{author_id}} = $rec_num;

print ". ";

$rec_num++;
}

print "Done Authors";


print "Events: ";
my $sth = $dbh->prepare("SELECT * FROM event");
$sth->execute();
while (my $ref = $sth -> fetchrow_hashref()) {

unless ($ref->{event_crdate}) { $ref->{event_crdate} = time; }

$node->{nid} = $rec_num;
$node->{vid} = $rec_num;
$node->{type} = "content_event";
$node->{title} = $ref->{event_title};
$node->{uid} = 1;
$node->{status} = 1;
$node->{created} = $ref->{event_crdate};
$node->{changed} = $ref->{event_crdate};
$node->{promote} = 1;
$node->{moderate} = 0;
$node->{sticky} = 0;
$vars->{idfield} = &db_insert($dbh,"node",$node);

$node_content_event->{vid} = $rec_num;
$node_content_event->{nid} = $rec_num;
$node_content_event->{field_event_type_value} = $ref->{event_type};
$node_content_event->{field_location_value} = $ref->{event_location};
$node_content_event->{field_start_date_value} = $ref->{event_start};
$node_content_event->{field_end_date_value} = $ref->{event_end};
$vars->{idfield} = &db_insert($dbh,"node_content_event",$node_content_event);

$node_revisions->{vid} = $rec_num;
$node_revisions->{nid} = $rec_num;
$node_revisions->{uid} = 1;
$node_revisions->{title} = $ref->{event_title};
$node_revisions->{timestamp} = $ref->{event_crdate};
$node_revisions->{format} = 0;
$vars->{idfield} = &db_insert($dbh,"node_revisions",$node_revisions);

$eventcodes->{$ref->{author_id}} = $rec_num;

print ". ";

$rec_num++;
}

print "Done Events";



my $sth = $dbh->prepare("SELECT * FROM journal");
$sth->execute();
while (my $ref = $sth -> fetchrow_hashref()) {

unless ($ref->{journal_crdate}) { $ref->{journal_crdate} = time; }

$node->{nid} = $rec_num;
$node->{vid} = $rec_num;
$node->{type} = "content_journal";
$node->{title} = $ref->{journal_title};
$node->{uid} = 1;
$node->{status} = 1;
$node->{created} = $ref->{journal_crdate};
$node->{changed} = $ref->{journal_crdate};
$node->{promote} = 1;
$node->{moderate} = 0;
$node->{sticky} = 0;
$vars->{idfield} = &db_insert($dbh,"node",$node);

$node_content_journal->{vid} = $rec_num;
$node_content_journal->{nid} = $rec_num;
$vars->{idfield} = &db_insert($dbh,"node_content_journal",$node_content_journal);

$node_revisions->{vid} = $rec_num;
$node_revisions->{nid} = $rec_num;
$node_revisions->{uid} = 1;
$node_revisions->{title} = $ref->{journal_title};
$node_revisions->{timestamp} = $ref->{journal_crdate};
$node_revisions->{format} = 0;
$vars->{idfield} = &db_insert($dbh,"node_revisions",$node_revisions);

$journalcodes->{$ref->{journal_id}} = $rec_num;

print ". ";

$rec_num++;
}

print "Done Journals";



my $sth = $dbh->prepare("SELECT * FROM person");
$sth->execute();
$unum = 10;
while (my $ref = $sth -> fetchrow_hashref()) {

unless ($ref->{person_crdate}) { $ref->{person_crdate} = time; }

$users->{uid} = $unum;
$users->{name} = $ref->{person_title};
$users->{pass} = "6a5d8b78b0ac67c4396d56875aaf438c";
$users->{mail} = $ref->{person_email};
$users->{created} = $ref->{person_crdate};
$users->{status} = 1;
$users->{init} = $ref->{person_email};
$vars->{idfield} = &db_insert($dbh,"users",$users);

if ($ref->{person_city}) {
$profs1->{fid} = 1;
$profs1->{uid} = $unum;
$profs1->{value} = $ref->{person_city};
$vars->{idfield} = &db_insert($dbh,"profile_values",$profs1);
}

if ($ref->{person_province}) {
$profs2->{fid} = 2;
$profs2->{uid} = $unum;
$profs2->{value} = $ref->{person_province};
$vars->{idfield} = &db_insert($dbh,"profile_values",$profs2);
}

if ($ref->{person_country}) {
$profs3->{fid} = 3;
$profs3->{uid} = $unum;
$profs3->{value} = $ref->{person_country};
$vars->{idfield} = &db_insert($dbh,"profile_values",$profs3);
}

if ($ref->{person_organization}) {
$profs4->{fid} = 4;
$profs4->{uid} = $unum;
$profs4->{value} = $ref->{person_organization};
$vars->{idfield} = &db_insert($dbh,"profile_values",$profs4);
}

if ($ref->{person_html}) {
$profs5->{fid} = 5;
$profs5->{uid} = $unum;
$profs5->{value} = $ref->{person_html};
$vars->{idfield} = &db_insert($dbh,"profile_values",$profs5);
}

if ($ref->{person_weblog}) {
$profs6->{fid} = 6;
$profs6->{uid} = $unum;
$profs6->{value} = $ref->{person_weblog};
$vars->{idfield} = &db_insert($dbh,"profile_values",$profs6);
}

if ($ref->{person_xml}) {
$profs7->{fid} = 7;
$profs7->{uid} = $unum;
$profs7->{value} = $ref->{person_xml};
$vars->{idfield} = &db_insert($dbh,"profile_values",$profs7);
}

if ($ref->{person_foaf}) {
$profs8->{fid} = 8;
$profs8->{uid} = $unum;
$profs8->{value} = $ref->{person_foaf};
$vars->{idfield} = &db_insert($dbh,"profile_values",$profs8);
}

$usercodes->{$ref->{person_id}} = $unum;
$usernames->{$ref->{person_id}} = $ref->{person_title};
print ". ";

$unum++;
}

print "Done Persons";



print "Posts: ";
my $sth = $dbh->prepare("SELECT * FROM post");
$sth->execute();
while (my $ref = $sth -> fetchrow_hashref()) {

next unless ($ref->{post_type} eq "link");
unless ($ref->{post_crdate}) { $ref->{post_crdate} = time; }

$node->{nid} = $rec_num;
$node->{vid} = $rec_num;
$node->{type} = "content_post";
$node->{title} = $ref->{post_title};
$node->{uid} = 1;
$node->{status} = 1;
$node->{created} = $ref->{post_crdate};
$node->{changed} = $ref->{post_crdate};
$node->{promote} = 1;
$node->{moderate} = 0;
$node->{sticky} = 0;
$vars->{idfield} = &db_insert($dbh,"node",$node);

$node_content_post->{vid} = $rec_num;
$node_content_post->{nid} = $rec_num;
$vars->{idfield} = &db_insert($dbh,"node_content_post",$node_content_post);

$node_revisions->{vid} = $rec_num;
$node_revisions->{nid} = $rec_num;
$node_revisions->{uid} = 1;
$node_revisions->{title} = $ref->{post_title};
$node_revisions->{timestamp} = $ref->{post_crdate};
$node_revisions->{format} = 0;
$vars->{idfield} = &db_insert($dbh,"node_revisions",$node_revisions);

$postcodes->{$ref->{post_id}} = $rec_num;

print ". ";

$rec_num++;
}

print "Done Posts";


print "Publications: ";
my $sth = $dbh->prepare("SELECT * FROM publication");
$sth->execute();
while (my $ref = $sth -> fetchrow_hashref()) {

unless ($ref->{publication_crdate}) { $ref->{publication_crdate} = time; }

$node->{nid} = $rec_num;
$node->{vid} = $rec_num;
$node->{type} = "content_publication";
$node->{title} = $ref->{publication_title};
$node->{uid} = 1;
$node->{status} = 1;
$node->{created} = $ref->{publication_crdate};
$node->{changed} = $ref->{publication_crdate};
$node->{promote} = 1;
$node->{moderate} = 0;
$node->{sticky} = 0;
$vars->{idfield} = &db_insert($dbh,"node",$node);

$node_content_publication->{nid} = $rec_num;
$node_content_publication->{vid} = $rec_num;
$node_content_publication->{field_category_value} = $ref->{publication_category};
$node_content_publication->{field_details_value} = $ref->{publication_catdetails};
$node_content_publication->{field_volume_value} = $ref->{publication_volume};
$node_content_publication->{field_pages_value} = $ref->{publication_pages};
$node_content_publication->{field_pub_type_value} = $ref->{publication_type};
$vars_content_publication->{idfield} = &db_insert($dbh,"node_content_publication",$node_content_publication);

$node_revisions->{vid} = $rec_num;
$node_revisions->{nid} = $rec_num;
$node_revisions->{uid} = 1;
$node_revisions->{title} = $ref->{publication_title};
$node_revisions->{timestamp} = $ref->{publication_crdate};
$node_revisions->{format} = 0;
$vars->{idfield} = &db_insert($dbh,"node_revisions",$node_revisions);

$publicationcodes->{$ref->{post_id}} = $rec_num;

print ". ";

$rec_num++;
}

print "Done Publications";

print "Topics: ";
my $sth = $dbh->prepare("SELECT * FROM topic");
$sth->execute();
while (my $ref = $sth -> fetchrow_hashref()) {

unless ($ref->{topic_crdate}) { $ref->{topic_crdate} = time; }

$node->{nid} = $rec_num;
$node->{vid} = $rec_num;
$node->{type} = "content_topic";
$node->{title} = $ref->{topic_title};
$node->{uid} = 1;
$node->{status} = 1;
$node->{created} = $ref->{topic_crdate};
$node->{changed} = $ref->{topic_crdate};
$node->{promote} = 1;
$node->{moderate} = 0;
$node->{sticky} = 0;
$vars->{idfield} = &db_insert($dbh,"node",$node);

$node_content_topic->{vid} = $rec_num;
$node_content_topic->{nid} = $rec_num;
$node_content_topic->{field_where_value} = $ref->{topic_where};
$node_content_topic->{field_topic_type_value} = $ref->{topic_type};
$vars_content_topic->{idfield} = &db_insert($dbh,"node_content_topic",$node_content_topic);

$node_revisions->{vid} = $rec_num;
$node_revisions->{nid} = $rec_num;
$node_revisions->{uid} = 1;
$node_revisions->{title} = $ref->{topic_title};
$node_revisions->{timestamp} = $ref->{topic_crdate};
$node_revisions->{format} = 0;
$vars->{idfield} = &db_insert($dbh,"node_revisions",$node_revisions);

$topiccodes->{$ref->{post_id}} = $rec_num;

print ". ";

$rec_num++;
}

print "Done Topics";




print "Preparing Comments";

print "Comments: ";
$cid = 1;
my $sth = $dbh->prepare("SELECT * FROM post");
$sth->execute();
while (my $ref = $sth -> fetchrow_hashref()) {

next unless ($ref->{post_type} eq "comment");
unless ($ref->{post_crdate}) { $ref->{post_crdate} = time; }

$comment->{cid} = $cid;
$comment->{nid} = $postcodes->{$ref->{post_thread}};
$comment->{uid} = $usercodes->{$ref->{post_creator}};
$comment->{subject} = $ref->{post_title};
$comment->{comment} = $ref->{post_description};
$comment->{hostname} = $ref->{post_crip};
$comment->{timestamp} = $ref->{post_crdate};
$comment->{name} = $usernames->{$ref->{post_creator}};
$vars->{idfield} = &db_insert($dbh,"comments",$comment);

print ". ";

$cid++;
}

print "Done Comments";




print "Preparing Data Type Contents:";

print "Articles: ";
my $sth = $dbh->prepare("SELECT * FROM post");
$sth->execute();
while (my $ref = $sth -> fetchrow_hashref()) {
next unless ($ref->{post_type} eq "article");
$rec_num = $postcodes->{$ref->{post_id}};
#print "Post ID is $ref->{post_id}and record number is $rec_num
";

&do_contents("post","article",$rec_num,$ref);
print ". ";
}

print "Done Articles";



print "Authors: ";
my $sth = $dbh->prepare("SELECT * FROM author");
$sth->execute();
while (my $ref = $sth -> fetchrow_hashref()) {
$rec_num = $authorcodes->{$ref->{author_id}};
&do_contents("author","author",$rec_num,$ref);
print ". ";
}

print "Done Authors";


print "Events: ";
my $sth = $dbh->prepare("SELECT * FROM event");
$sth->execute();
while (my $ref = $sth -> fetchrow_hashref()) {
$rec_num = $eventcodes->{$ref->{event_id}};
&do_contents("event","event",$rec_num,$ref);
print ". ";
}

print "Done Events";


print "Journals: ";
my $sth = $dbh->prepare("SELECT * FROM journal");
$sth->execute();
while (my $ref = $sth -> fetchrow_hashref()) {
$rec_num = $journalcodes->{$ref->{journal_id}};
&do_contents("journal","journal",$rec_num,$ref);
print ". ";
}

print "Done Journals";

print "Posts: ";
my $sth = $dbh->prepare("SELECT * FROM post");
$sth->execute();
while (my $ref = $sth -> fetchrow_hashref()) {
next unless ($ref->{post_type} eq "link");
$rec_num = $postcodes->{$ref->{post_id}};
&do_contents("post","post",$rec_num,$ref);
print ". ";
}

print "Done Posts";

print "Publicationss: ";
my $sth = $dbh->prepare("SELECT * FROM publication");
$sth->execute();
while (my $ref = $sth -> fetchrow_hashref()) {
$rec_num = $publicationcodes->{$ref->{publication_id}};
&do_contents("publication","publication",$rec_num,$ref);
print ". ";
}

print "Done Publications";

print "Topics: ";
my $sth = $dbh->prepare("SELECT * FROM topic");
$sth->execute();
while (my $ref = $sth -> fetchrow_hashref()) {
$rec_num = $topiccodes->{$ref->{topic_id}};
&do_contents("topic","topic",$rec_num,$ref);
print ". ";
}

print "Done Topics";

print "Updating Sequences";
$sth = $dbh->prepare("UPDATE sequences SET id = $rec_num WHERE name = 'node_nid'");
$sth->execute();
$sth = $dbh->prepare("UPDATE sequences SET id = $rec_num WHERE name = 'node_revisions_vid'");
$sth->execute();
$sth = $dbh->prepare("UPDATE sequences SET id = $cid WHERE name = 'comments_cid'");
$sth->execute();
$sth = $dbh->prepare("UPDATE sequences SET id = $unum WHERE name = 'users_uid'");
$sth->execute();


if ($dbh) { $dbh->disconnect; }


exit;

sub do_contents {

my $tbl = shift;
my $tbout = shift;
my $rec_num = shift;
my $ref = shift;

# $dc++; if ($dc > 100) { exit; }
#print "do_contents: $tbl $rec_num
";

$field = $tbl."_author";
if ($ref->{$field}) {
$f0->{vid} = $rec_num;
$f0->{nid} = $rec_num;
$f0->{field_aut_value} = $ref->{$field};
$vars->{idfield} = &db_insert($dbh,"node_data_field_aut",$f0);
}

$field = $tbl."_journal";
if ($ref->{$field}) {
$f10->{vid} = $rec_num;
$f10->{nid} = $rec_num;
$f10->{field_jou_value} = $ref->{$field};
$vars->{idfield} = &db_insert($dbh,"node_data_field_jou",$f10);
}

$field = $tbl."_authorid";
if ($ref->{$field}) {
$author_nid = $authorcodes->{$ref->{$field}};
#print "$field - $ref->{$field} - $author_nid
";
$f1->{vid} = $rec_num;
$f1->{nid} = $rec_num;
$f1->{field_author_nid} = $author_nid;
$vars->{idfield} = &db_insert($dbh,"node_data_field_author",$f1);
}

$field = $tbl."_crdate";
if ($ref->{$field}) {
$f2->{vid} = $rec_num;
$f2->{nid} = $rec_num;
$f2->{field_crdate_value} = $ref->{$field};
$vars->{idfield} = &db_insert($dbh,"node_data_field_crdate",$f2);
}

$field = $tbl."_description";
if ($ref->{$field}) {
$f3->{vid} = $rec_num;
$f3->{nid} = $rec_num;
$f3->{field_description_value} = $ref->{$field};
$f3->{field_description_format} = 1;
$vars->{idfield} = &db_insert($dbh,"node_data_field_description",$f3);
}

$field = $tbl."_email";
if ($ref->{$field}) {
$f4->{vid} = $rec_num;
$f4->{nid} = $rec_num;
$f4->{field_email_email} = $ref->{$field};
$vars->{idfield} = &db_insert($dbh,"node_data_field_email",$f4);
}

$field = $tbl."_event";
if ($ref->{$field}) {
$event_nid = $eventcodes->{$ref->{$field}};
$f5->{vid} = $rec_num;
$f5->{nid} = $rec_num;
$f5->{field_event_nid} = $event_nid;
$vars->{idfield} = &db_insert($dbh,"node_data_field_event",$f5);
}

$field = $tbl."_id";
if ($ref->{$field}) {
$f6->{vid} = $rec_num;
$f6->{nid} = $rec_num;
$f6->{field_id_value} = $ref->{$field};
$vars->{idfield} = &db_insert($dbh,"node_data_field_id",$f6);
}

$field = $tbl."_link";
if ($ref->{$field}) {
$f7->{vid} = $rec_num;
$f7->{nid} = $rec_num;
$f7->{field_link_value} = $ref->{$field};
$vars->{idfield} = &db_insert($dbh,"node_data_field_link",$f7);
} else {

$field = $tbl."_url";
if ($ref->{$field}) {
$f7->{vid} = $rec_num;
$f7->{nid} = $rec_num;
$f7->{field_link_value} = $ref->{$field};
$vars->{idfield} = &db_insert($dbh,"node_data_field_link",$f7);
}
}

$field = $tbl."_journalid";
if ($ref->{$field}) {
$journal_nid = $journalcodes->{$ref->{$field}};
$f8->{vid} = $rec_num;
$f8->{nid} = $rec_num;
$f8->{field_journal_nid} = $journal_nid;
$vars->{idfield} = &db_insert($dbh,"node_data_field_journal",$f8);
}

$field = $tbl."_status";
if ($ref->{$field}) {
$f9->{vid} = $rec_num;
$f9->{nid} = $rec_num;
$f9->{field_link_value} = $ref->{$field};
$vars->{idfield} = &db_insert($dbh,"node_data_field_status",$f9);
}

return;
}

sub db_open {

my ($dsn,$user,$password) = @_;
my $dbh = DBI->connect($dsn, $user, $password);
unless ($dbh) { print "Database connect error: $! \n"; return; }
# if ($dbh) { $dbh->trace(2,"dberror.txt"); }
return $dbh;
}

sub db_insert { # Inserts record into table from hash

my $dbh = shift;
unless ($dbh) { print "Database handler not initiated"; return; }
my $table = shift;
unless ($table) { print "table not specified for input"; return; }
my $input = shift;
unless ($input) { print "No data provided on insert in $table"; return; }

# die "Unsupported data type specified to insert" unless (ref $input eq 'HASH' || ref $input eq 'Link' || ref $input eq 'Feed');

my $data= &db_prepare_input($dbh,$table,$input);

my $sql = "INSERT INTO $table ";
my(@sqlf, @sqlv, @sqlq) = ();

for my $k (sort keys %$data) {
push @sqlf, $k;
push @sqlq, '?';
push @sqlv, $data->{$k};
}
$sql .= '(' . join(', ', @sqlf) .') VALUES ('. join(', ', @sqlq) .')';
#if ($table eq "node_data_field_author") { print "$sql -- ",@sqlv,"

"; }


my $sth = $dbh->prepare($sql);
$sth->execute(@sqlv);

return $dbh->{'mysql_insertid'};

# Adapted from SQL::Abstract by Nathan Wiger
}


sub db_prepare_input { # Filters input hash to contain only columns in given table

my ($dbh,$table,$input) = @_;
my $data = ();

my @columns = &db_columns($dbh,$table); # Get a list of columns safeguard data input
foreach my $ikeys (keys %$input) { # Clean input for save
next unless ($input->{$ikeys}); # - no blank fields
next if ($ikeys =~ /_id$/i); # - do not change ID
next unless (&index_of($ikeys,\@columns) >= 0); # - input column must exist
$data->{$ikeys} = $input->{$ikeys}; # Transfer to input hash
#$data->{$ikeys} = &demoronise($data->{$ikeys}); # Fix non-standard character input
}

return $data;

}

sub db_columns {

my ($dbh,$table) = @_;
my @columns = ();
my $showstmt = "SHOW COLUMNS FROM $table";
my $sth = $dbh -> prepare($showstmt);
$sth -> execute();
while (my $showref = $sth -> fetchrow_hashref()) { push @columns,$showref->{Field}; }

unless (@columns) { print "Can't find any columns for $table"; return; }
return @columns;
}

sub index_of {

# Get item and array from input
my ($item,$array) = @_;

# Initialize counter
my $index_count = 0;
# For each item in the array
foreach my $i (@$array) {
# Return the counter value if it matches item
if ($item eq $i) { return $index_count; }

# Increment the counter
$index_count++;
}

# Return -1 if no match is returned
return "-1";
}

Tuesday, December 05, 2006

A New Website, Part Twelve - Node Fields

Well... I've been working on posts and have noticed that the whole Drupal database has changed form as I've added content types. Here's what the node tables look like now:

node
node_access
node_comment_statistics
node_content_article
node_content_author
node_content_event
node_content_journal
node_content_post
node_content_quote
node_counter
node_data_field_author
node_data_field_crdate
node_data_field_description
node_data_field_email
node_data_field_event
node_data_field_id
node_data_field_link
node_field
node_field_instance
node_revisions
node_type_content

So what's happening? Node data always gets entered into node and node_revisions, as before. But for content types, one of two things happens.

If a content field, such as 'description', is used in two or more content types, then a special data table, node_data_field_description, is created for that field, and the data stored there.

Otherwise, the data is stored in a field reserved for the data type itself, such as node_content_post.

No great disaster, I'm just glad I found this out now. It means some changes to the script, nothing major.

I also discovered that my post database is 17 megabytes large. To load it, I saved it as an SQL file, uploaded by FTP, then ran the command:

mysql -u username -p databasename < post.sql

Worked like a charm.

At Last!

Yay! I am listening to crisp and clear CBC audio in my office for the first time ever! Thank you for this - now I don't need to listen to country music from Australia any more!

-- Stephen

CBC Input wrote:
Dear Stephen,

Thank you for your email. Just to let you know CBC.ca is currently testing the streaming of Ogg Vorbis, an open, free audio codec. It can be used with Linux, MAC and Windows. Please go to our 'Listen To Radio' at http://www.cbc.ca/listen/index.html and scroll down the right-hand column to the
Ogg link. Windows users must download the free WinAmp 5 (full) media player and others can just download the Ogg page.

We would appreciate your testing this system and letting us know how you like it.

Thanks again for taking the time to contact us.

Parker Bishop
Communications Assistant
CBC Audience Relations

The Form of Informal - 2

Responding to Tony Karrer, More on the Form of Informal

What do you think the meaning of the word 'dictating' is?

"How do you aim at performance objectives, provide appropriate support, structure and form without dictating to some level?"

I can 'aim at performance objectives' in a variety of ways, some dictatorial, others not:

- I can suggest that the company (or society) needs more widget experts, for example, or
- I can require that all employees undergo widget training

Similarly, given that for whatever reason a person has decided to learn about widgets:

- I can make available some resources on widgets
- I can provide some just-in-time learning support for people using widgets
- I can give salary bonuses for people who demonstrate they have passed a widget course
- I can assign an online course that all staff must complete
- I can hold mandatory widget training sessions

"No one says its structure free, but how much structure is allowed before it becomes too much to be informal? There's some kind of spectrum here with all sorts of shades."

The presumption here is that 'structure' is some sort of monodimensional property, akin to (say) 'complexity'. But it is not. Something can be very structured in one dimension and very unstructured in another.

For example: viewed a certain way, a forest is very unstructured. Nobody is organizing anything. Trees grow whether they will, deer and other woodland creatures wander about with no guidance whatsoever. But viewed another way, a forest is very structured. It is an instance of a complex ecosystem. Patterns are repeated in the shapes of the trees, the shapes of the leaves.

The point here is: 'structure' is not a 'specturum' per se. Rather, some types of structure tend toward 'informal', while other types of structure tend toward 'formal'.

I'll come back to that.

"What's interesting is that the moment you begin to understand the form that informal learning took and provide support for that kind of learning in the future you start down the path of dictating solutions."

Again, this simply doesn't follow, unless you have a very unorthodox understanding of the meaning of the word 'dictating'.

The term 'dictating', as normally understood, entails some sense of enforcement. To 'dictate' that something will be done is to require it to be done, to impose an obligation on someone that it be done, and hence, concurrent with the concept of enforcement, is related to the concepts of power and control.

But not all structure is the result of, nor requires, power and control. This is the fallacy I've been trying to get at.

Let's call this the 'teleological fallacy' - the presumption that, where there is order, there must be an organizer, someone who creates and manages, through some process of authority, this order. A 'God' of the training room 'design', if you will.

"Providing job aids was one example that Michael McGinnis cited. The first time a person learned how to do that task/job it was likely through someone showing them how to do it. Once they put it in a job aid - it feels more formal on the spectrum. So maybe it's not a paradox, maybe its a spectrum."

The mere transfer of some 'showing' into some 'job aid' does not make something more formal. Some very formal instruction - such as, say, by a drill sergeant - can involve 'showing'. And some very informal instruction - such as a 'how-to' manual - can be a job aid.

So why does it 'feel more formal (in, at least, some cases)? Not because it has been given structure, but because the structure has changed. Some types of structure are formal, others are informal.

"But, it feels a lot like what Artifical Intelligence faces - informal is a bit mysterious and putting structure to it makes it feel more formal and much less mysterious."

Perhaps by 'structure' here you mean 'rule-based'?

Again - informal does not mean 'having no structure'. Rather, 'informal' means having a different kind of structure. One that is, among other things:
- not dictatorial
- not organized or managed by an organizer
- not rule-based

"Of course, I personally am not that concerned with the definition of the term nor really even the paradox as I am in understanding the next level of informal learning: what kinds of guidance, what kinds of support, how can be provided, that ultimately lead us to accomplishing our performance objectives?"

Right. And neither am I, because it is in attempting to define the term that we get right into the sort of fruitless and endless debates that formal semanticism so often entail.

But...

It is still relevant to ask, what sort of structure tends to characterize the informal, and what sort of structure tends to characterize the formal? Structure that is not dictatorial, not organized, and not rule-based, to be sure. But what sort of structures look like that?

In a slogan: networks, as opposed to hierarchies.

I have elsewhere characterized the properties of systems that tend to be networks, as opposed to hierarchies, via their having following structural properties:
- decentralized.
- distributed.
- disintermediated.
- disaggregated.
- dis-integrated.
- democratic.
- dynamic.

Sorry about the cutesy list of Ds.

The sixth condition, 'democratic', constitutes what I call the semantic condition, and is constituted of four major elements:
- autonomy
- diversity
- openness
- connectedness

These (and conditions similar to these - I am not wedded to any particular characterization of these conditions) constitute a metric that distinguishes formal from informal. Things that have more of these properties are more informal, and vice versa (this will not be a stricly linear progression, which is why it is not merely a spectrum; take away even one characteristic - 'autonomy', say, and you instantly convert a system that was very informal to something very formal. The way a party becomes something very different entirely the moment someone says you have to attend.

But the main point here is that, within these parameters, there is a great deal of room (and expectation) of organization. The concept of 'distributed', for example, by its very nature assumes the existence of communications protocols between the parts. And the same for the other conditions.

When we look at this sort of model, we can see how unsatisfactory discussion akin to "Providing job aids was one example..." can be. We ask, immediately, what sort of job aid? Is it something employees were required to use (like, say, project template forms)? Is it something that needs to be scheduled? Managed? Produced from a central depot by a team of experts? All of these effect whether it can be characterized as 'formal' or 'informal'.

Different types of technology, different types of learning materials and different types of pedagogical practice may be characterized as more or less informal according to these criteria. Not based on the simplistic assessment of whether they are 'structured', but rather, on the much more insightful determination of how they are structured.

Monday, December 04, 2006

The Form of Informal

Clark Aldrich, as cited by Tony Karrer, writes, "Can one criticize formal learning models in a book? Isn't a book the epitome of what one is suggesting is the wrong model?"

This is a pretty equivocal discussion. The words 'formal' and 'informal' are used in distinct senses.

In one sense, something is 'formal' or 'informal' according to whether it forms some part of a recognition infrastructure - the system of classes, programs and institutions that constitute our certificate and degree granting structure. Thus, when one has 'formal' qualifications, it is in reference to this infrastructure.

In a second sense, something is 'formal' if it is derived or represented in or as an abstract structure. In this way, systems or methods of doing things are 'formal', as are structures, 'forms' and other abstract entities. Thus, when we study 'formal' logic, we study abstract principles of reason, mechanisms for successful argumentation, truth tables and truth preservation, and the like.

Clearly there is a very large difference between having a method of doing something and having some educational infrastructure regulate and recognize the doing of something.

What complicates the matter is that the criticism - 'how can somebody write a book about informal learning' misses the mark in both senses. Because we have to ask - in what sense is a book formal?

The most that we could do is to try to adduce some sort of third sense of formal - something based on the length, or perhaps the book being organized into sections and chapters, perhaps, or even something based on the book being written in a language, which is at heart sequential. But neither segmentation nor sequence is sufficient to make something 'formal'.

I have over time lost patience with people who criticize informal learning on the ground that, if it is informal, it has no structure whatsoever. Articles like 'Informal Learning is Too Important to Leave to Chance,' I might add, fall into that category.

The mere fact that the learner is not being directed by some teacher or educational institution does not entail that there can and must be no structure whatsoever. nobody equates 'informal learning' with 'structure-free learning'.

The test for this is simple: ask any person who has learned something informally - whether it is how to program computers or how to tune and engine - how they learned. If they have any answer, then there was some structure to the learning. If a person says, "well I followed the discussion lists and then I looked up what I needed on Google," there's a structure. If he says, "Well I watched my uncle Fred tune a few engines, then I tried it while he watched," there's a structure.

What makes informal learning different from formal learning is not that it is formless, but rather, it that it is conducted outside the domain of the formal education infrastructure, with the associated and not trivial implication that it is managed by the learner, and not the professor or institution.

That's why a statement like 'too important to be left to chance' is so misleading. It implies that there is no reason why a person (whether an employee or a student) might choose this or that informal learning method. It implies that nothing can be done to support this person, to suggest some structures or mechanisms, to improve their likely outcome. It assumes that, unless we control this person, the outcome is 'by chance'.

But that's ridiculous. It is one thing to say, "I'm personally much more confident if I have a set of performance objectives that I can use to derive learning objectives and skill development opportunities around. I want to put structure in place that guides the learner along the way." It is quite another to say that the learner (a) must attend your class, and (b) must adhere to your learning outcomes and learning methodologies.

When I have characterized the distinction between formal and informal learning, I have done it this way: by saying, if you can walk out of the room, or change the shape of the discussion, or skip an activity, without (academic or other) sanction or penalty, then it's informal learning.

People can publish a book and still support informal learning. After all, picking up a book, reading it on your own time, and consulting it as necessary, is the epitome of informal learning. No classes, no structure, no tests, no grades, no degrees. Reading a book written by James Joyce is not somehow 'more informal' than reading a book by Jay Cross.

From my perspective, I see more of a tension when people refer to books as being authoritative on the internet.

For example, Dave Warlick did something to one of my comments recently to which I took objection, and when he asked me to justify my complaint he asked, "What book is it written in?" As though this would be authoritative.

When I replied, the internet, it was taken by some readers to be flippant or even rude. Why would I not provide a citation? But what citation would be better than the internet itself when addressing a question of what is or is not done on the internet? How would what is written in a book actually outweigh what people do online?

In certain circles, publishing a book still confers some authority upon its author, as though cozying up to some publisher (and signing away your copyright) somehow grants you some sort of status. But I don't agree.

We are at the cusp of a division between the age when people gained authority through what they printed on paper and when people gain authority through what they print online. It will not be a quick transition, because it is easy to recognize when somebody publishes a book, and much harder to recognize when they publish a quality website.

But as time goes by, and when we realize that Paris Hilton has published one too many books for our liking, or that some online writer has published one too few, this transition, from being anointed an 'expert' by the publishing industry, to being anointed an 'expert' by a community (of practice), will become easier.

A New Website, Part Eleven - Content Construction

Still working on the data transfer...

Author

This transfer went relatively smoothly.

It wasn't clear to me at first how the system would support multiple authors, but it turns out that CCK does it quite nicely. It is important to define your properties in the right order. In other words, to define the 'authors' table first, and define 'post' later, so that you can select the author table from a list when defining how to designate the post author.

Journal

The list of journals loaded easily, even more so than the loading of author because it proceeded in the exact same way.

It is worth noting that the node counter in Drupal does not increment authomatically, which means that after a bunch of records are inserted you have to change the counter manually.

There are several counters:
- menu_mid
- users_uid
- node_nid
- node_revisions_vid
- comments_cid

The naming system is pretty intuitive; the first part is the name of the table, and the second is the name of the field. Thus, after inputting a bunch of authors and journals, I had to insert a new value into each of node_nid and node_revisions_vid. Otherwise, any new content would attempt to over-write an existing record, which generates (as it should) an error.

Event

Event is a content type I devoted a fair amount of time to in my previous system and haven't quite got down. But I know where I want it to head.

First of all, how do define events: I needed to decide between the 'Calendar' module and the 'CCK' module.

I first created an 'Event' data type in CCK. This worked reasonably well, however, my date strings had to be plain integers. No really problem, since I'm using unix-style dates. However, these need to display nicely as well, and also, on my existing site I had a nice popup calendar I could click.

How about the Calendar module? First I had to find it - once again, Drupal's awful search made what should have been a one-minute job a ten minute exercise in frustration. Once I'm at the page I realize that there really isn't any information.

I don't understand the thinking behind the module pages in Drupal. Why are there no links to examples (or screen shots, or whatever) so I can see what they look like and what they do. And why doesn't the 'support forum' link to a module-specific support forum?

Anyhow. Given that 'calendar' requires the 'views' module, but not 'CCK', I conclude that calendar data cannot be associated with the other type of data I want to create. So I decide to go the CCK route.

Why is this important for me? Because in addition to content metadata, I foresee an environment that will eventually support event metadata. This in turn means that we will want to associate content metadata with event metadata, and vice versa. So I need content association.

What also helped is that while I was browsing around looking for the Calendar module I came across the various CCK field type modules. Very handy, especially the CCK date module. This module will even give me little calendar popups if the Javascript Tools module is installed (Javascript Tools also have a number of other utilities, such as form checking, AJAX support and columns).

Both modules installed without incident, but when I redesigned the Event content type to use the new date module the nice Javascript widget didn't pop up. I had to actually read the README file before I realized that after installing the Javascript Tools module you have to enable not only the main module but the specific function module, in this case, jscalendar.

While downloading CCK content types I also decided to download the email module; this would give me some form-checking on email address submissions.

Also worth mentioning when adding new fields to a custom content type: you should reuse the fields you've already created, especially if you are using the same field name. I noticed that I had three distinct 'Description' fields defined, which CCK dutifully named 'Description-0', 'Description-1', etc. It turned out to be a lot easier to simply select from the list, with no confusion of data.

Users

Transferring the user database is obviously very different from the rest, since it's not just another common data type, but rather, is connected with logins, mailing list subscriptions, and access rights.

In Drupal, you can extend the user profile by adding any number of fields. These fields are grouped according to category, and these categories display as tabs when the user opts to 'Edit' their profile.

I began by looking at the mailing list modules. As mentioned, my server was using the ezmlm, so I used the ezmlm module. However, on learning that CSoft is switching over (presumably this week) to mailman, I decided to remove the ezmlm module and install the mailman manager.

The way the mailing list managers work is via email commands, and that's how Drupal interacts with them. When you install the manager you specify the list address. Drupal then sends commands (accompanied by a password) back and forth to the list manager, meaning that the user can manage their subscriptions entirely via Drupal.

As with ezmlm, therefore, what this means is that I will need to populate the mailman list with existing subscriptions, then populate Drupal with existing user profiles, which include email addresses, and then coax Drupal into recognizing those subscriptions via the email addresses. This is very similar to how my existing website works (subscription lists are maintained separately from personal profiles).

Since mailman isn't ready for prime time on CSoft, I decided to turn my attention to the users database.

Drupal supports the following basic user profile in the 'users' table:


int(10)
UNSIGNED No 0








varchar(60) utf8_general_ci
No









varchar(32) utf8_general_ci
No









varchar(64) utf8_general_ci
Yes









tinyint(1)

No 0








tinyint(1)

Yes 0








tinyint(1)

Yes 0








varchar(255) utf8_general_ci
No









varchar(255) utf8_general_ci
No









int(11)

No 0








int(11)

No 0








int(11)

No 0








tinyint(4)

No 0








varchar(8) utf8_general_ci
Yes NULL








varchar(12) utf8_general_ci
No









varchar(255) utf8_general_ci
No









varchar(64) utf8_general_ci
Yes









longtext utf8_general_ci
Yes NULL








Most of these data elements are pretty intuitive and map one-to-one from my existing data.

Additional data elements are supported in the 'profile_fields' and 'profile_values' tables. 'profile_fields' defines the names and some other variables associated with each field, assigning each field a field number. 'profile_values' associates the field number with the actual values. This may seem complex, but doing it this way allows you to assign variables to each field (such as, say, the type of data accepted, or the name of the widget used to collect or display data) and to easily change the name of a field without changing the structure of data tables.

Fields are added through the Drupal administration panel, under 'settings - profile'. I created a number of fields to match my existing data:
- City
- Province or State
- Country
- Organization
- Web Page
- Weblog
- XML or RSS
- FOAF
- Status
- Mode
- Eformat
- ID Certificate

I then wrote a crosswalk that would populate not only the user table but also the profile_values table. Note also that the users_uid value in the sequences table also needs to be set, since the profile counter does not auto-increment.

Post

The heart of my website is the Post. On my site, the post acts in the way the node acts on Drupal, as the repository for a variety of different content types. In particular, there are three major types of Post on my site:
- links - the short items that populate OLDaily each day
- articles - the longer, blog-post pieces that I write
- comments - contributions from visitors to the site

The rough equivalent in Drupal are the 'story' and 'page' content types. Neither really matches my type of content. Additionally, comments in Drupal are treated as a separate content type entirely.

In both my system and in the Drupal system, comments are associated with posts (or other content items) via the post ID number (or the node ID number, in the case of Drupal). So it won't be that hard to preserve that association. Moreover, both posts and comments are associated with a user, where the user is the Drupal userid (and on my system, the corresponding person ID). So I can preserve authorship.

Additionally, posts that are links, on my system, are associated with authors and journals (or publishers). These had their own IDs on my system, and now have their own distinct node ID on the Drupal system. When crosswalking the author and journal data, I save these in a hash table, for example, $author{Drupal ID} = Old ID. This way I can easily convert from the old ID value to the new ID value.

That's it for today. More on posts tomorrow.