Designing E-Learning 3.0 in gRSShopper - 7

---

E-Learning 3.0 - Part 1 - Part 2 - Part 3 - Part 4 - Part 5 - Part 6 - Part 7 - Part 8 - Part 9 - Part 10 - Part 11 - Part 12 - Part 13

---

OK, I've put on some Cœur de pirate and am set for today's work.

Feeds

One of the key attributes of gRSShopper is the harvester. This application scans websites for web feeds which it then retrieves and displays to the user for reading and processing.

In a cMOOC one of the core student activities is to create blogs and submit their feed to the MOOC. I used to have this built into a general form submission function, but I got a lot of spam and other junk. It was also a weak point bad actors could attempt to exploit. So I've written a script, submit.cgi, which is isolated from the other scripts and fairly strict in what it will do.

So here's what it looks like right now:

Figure 80 - Basic feed submission form

Now this works - if you submit the feed, it submits it to the database. But I can't see it yet because of a bug in gRSShopper.

'Feed' is a data type, and there is a way to view feeds just like everything else:

Figure 81 - Feed Editor, Edit tab

If you create a new feed in gRSShopper, it will show up, no problem.

Figure 82 - Feed Editor, Classify tab

There are different ways to classify feeds, and this will be useful in the future some time (it was really useful for Moncton Free Press).

But what we're interested in for now is the harvester, which is accessed through the Harvest tab. Here it is:

Figure 83 - Feed Editor, Harvest tab

The harvester reflects some of the ambiguity of feeds.

A feed is a content source. For me, that means any content source, whether I'm actively harvesting it or not.

When I create a new blog post, I list the author, and I list the feed, always. Yes, I could distinguish between blog, magazine, newspaper, book, journal, and whatever, but I don't. This is part of what it means to be working with data - the physical incarnation of the data doesn't really matter (though it might be a useful way to classify them).

That means I have four types of feeds (and I've associated colours with them):

Unlinked - this is a feed that has no harvest URL. It's almost always a feed I've created manually in the process of creating a post. The harvester can't and won't harvest it.

On Hold - this is a feed that has a harvest URL, but I haven't approved it for harvesting. These are almost always feeds that have been submitted (through, say, submit.cgi) but which I haven't yet reviewed (I have found I need to review them because the submitted links are often wrong and need fixing - this was one of the most time-consuming problems of previous MOOCs).

Approved - this is a feed that I've approved (by clicking the 'Approved' option in teh Harvester tab). Once I've approved the feed, the harvester will harvest the feed.

Retired - this is a feed I've decided to stop harvesting. I don't simply delete the feed, because I might still have a lot of records associated with the feed (in gRSShopper, when you delete a record, all graph references to the record are also deleted, so there's no broken links, but we lose the information.)

Now when I list feeds , there's a way to filter the listing:

Figure 83 - List Feeds - search function

This is the bit that's broken. There's no way to view the orange feeds in order to review them. I need to fix the controls for the harvester, which is what I'll do today, because I really need this to work.

(time passes)


Somewhere in the record creation script I have a line that specifies that new feeds should be classified as 'O' (Orange, standing for 'On Hold'). This is fine, but since then I have been classifying feeds without feed_link values as 'B' (Brown, standing for 'Unlinked'). So I need to edit this - after I convert all the feeds marked 'O' in the database to 'B'.

(more time passes)

OK, I've edited the search form (the search itself hasn't been fixed yet):

Figure 84 - Feed Search

I need to add the filter that selects for one or more types, but for now, being able to view all feeds works fine. Also, the intent is to set the default to filter for Green (Active) feeds.

(still more time passes)

OK, I've edited the submit.cgi script to manage incoming feeds (this is actually something I've done a bunch of times over the years, but by breaking this off and making it its own thing maybe I can stop rewriting this part of the code).

Here's what I added:

1. A simple HTML page that will load the actual 'submit feed' form into an iFrame - this makes it a lot easier to keep this page up to date with template changes
2. A slightly modified form, allowing submission of blog web pages as well as feed pages (update: in the first 24 hours one person has already gotten this wrong, so...). Here it is:

Figure 85 - Submit Feed Form

3. Server-side script that makes sure you have submitted a value for title, web page, and feel link. I know this could be done with Javascript, and I should probably create a Javascript version of the fom, but this is what I have for now. Here's what the error looks like:

Figure 86 - Submit Feed Error Report

Note the nifty 'Go Back' button. This is a bit of Javascript magic and takes you back to your original form. Here's the Javascript:

Figure 87 - Javascript Back Button

I think I'll use this more, in other places.

4. Script to check whether the title, html or link already exist in the database. I don't want to allow duplicates.

This is tricky because people might want to update their information, but I don't want to allow just anyone to update information. It's especially tricky because I pre-added some feeds, so the feed owners won't be able to submit them.

Usually at this point systems offer some method to 'claim your feed', typically by having them insert some key or code into the feed itself. Alternatively, the system could ask you to prove your identity, either using a direct OAuth2 call to your feel (ie., to your blog host) (which most often isn't supported by the blog) or to have the feed point to a Twitter identity (or GitHub, or some other thing) which does support OAuth2, and have you verify that way (that's how IndieWeb does it).

I'm not sure how I want to go forward with this, so I'll leave it for now. Something for the roadmap.

So if I get a duplicate, I invite people to view the feed list (which I still need to create) and to check with me if there's something wrong.

5. Script to test the feed link provided and report back if there's an error. This saves me from deleting feeds with typos in the feed URL. 

6. Default the submitted feed to 'O' (On Hold)

7. Save it and send a confirming message.

(more time passes, and we're into the next day)

Now I need to generate the feed list page. I create the page using keywords, like I've done with previous pages:

Figure 88: Feed List page in the Page Editor

I've set this page to autopublish, so I don't need to be constantly refreshing it to get an updated list of feeds. It prints out to course_feeds.htm

But when a person submits a feed and needs to see right away what happened if there's an error, I can't just send them to the web page, because it only autopublishes once a day. And even if I published it more frequently, that wouldn't help. What I need is a live version of the page.

The way to access pages directly from the database is to use a page.cgi request. For http://el30.mooc.ca the URL would be as follows:

     http://el30.mooc.ca/cgi-bin/page.cgi?page=Course Feeds

However, this won't generate the 100% live page, because page.cgi prefers to use a locally cached version for requests like this (this is especially useful for things like posts and modules, etc., which are created once and might never change).

Note: the cache doesn't automatically update; something to fix.

So what you need to do is force page.cgi to give you the most up-to-date version. So we use the 'force' option.

     http://el30.mooc.ca/cgi-bin/page.cgi?force=yes&page=Course Feeds

This is the link I give to people when they're creating their feeds and need to see the most up-to-date page (note that using 'force' does not publish the page, nor even update the cache; it's used only to preview the page).

Having done that, I need to made sure I have a nice feed_summary view (there's one that comes with gRSShopper now, but it's a bit ugly). Here's what I created:

Figure 89 - feed_summary view in the View Editor

And here's the page that results:

Figure 90 - Course Feeds page

To finish this part of the work, I added a link to the 'Submit Feed' and 'Your Feeds' pages in the course contents box, and then republished a bunch of course pages so they'd be available right away.

A New Type of Post

I've been adding both participant posts and resource posts to the newsletter. but I've decided I don't want them mixed into the same list any more. So I've decided to create a new type of post - I'll call it a 'resource' post.

Just so we know, the default type of post is the 'link' post. That's the post with a title and a link and which is typically created by the harvester or when I create a new post with all the default settings.

If I wanted a different type of post, I would click one of the post_type options in the Publish tab in the Post Editor:

Figure 91 - Post types in the Publish tab in the Post Editor

The list is Link, Article, Comment, etc. You'll notice that 'Resource' is not an option.

So we'll make it an options. Options are stored in the Optlist (short for 'Option List') table. This table comes pre-defined in gRSShopper, but you can change it any way you want.

Optlist elements are named according to what they define. For example, if the table is 'post' and the option is 'type' then the corresponding Optlist is 'post_type' (one day I should just create a direct link to it in the editor. Something to fix).

Here's the post_type Optlist:

Figure 92 - Optlist Editor

The format is: name,value;name;value ...

It's pretty basic and one day I'll write a nice interface, but this works wekk for now (though I'm always forgetting whether name goes first or value goes first - usually I just use exactly the same thing so I don't need to remember).

So I'll just add my new Resource post type:

Figure 93 - Optlist Editor with Resource added as a data type

Now when I edit a post, the 'Resource' type is available to me. I inserted into the third place because options are displayed in order, and I wanted this one third. We see it there below:

Figure 94 - Post Editor with new Resource type added

Now in a page like the newsletter I'll just use two different keywords:

Figure 96 - Page editor using keywords to load different types of post

and this separates link posts from resource posts for me. Note that I'll have to create views for this new type of post: post_resource_email, for example.

Articles

As you can see from the image, I also decided to add an article. This is another type of post (maybe I should make 'Article' a different type of content entirely, but I didn't, and I don't feel like changing that). An article is distinct from other types of posts in that it makes greater use of the 'content' element. In gRSShopper you can type your articles right into the form, using the 'write' tab. Here it is:

Figure 97 - Page Editor, Write tab

Articles can be displayed using the post_article_html view. Here's the one I created for the E-Learning 3.0 course:

Figure 98 - post_article_html view in the View Editor

Notice how the date is formatted (format=nice) using the post creation date (post_crdate).

Notice the use of the post_content data element. This is the content you created in the Write tab. It's saved as text in the database and can be inserted wherever you want.

Also notice the use of keylist command to associate an article with a publication. A publication is something like a journal article, magazine column, whatever. The publication table contains this publication information. Nothing in the course has been published anywhere, so this keylist command does nothing. But if there were a published version of this article, the command would print the 'prefix', then the publication information (format=inpost) and then (if it were defined) the 'postfix'.

Here's what the article looks like on the page:

Figure 99 - Article page

I tweaked the post_link_email, post_resource_email and post_article_summary Views a bit, and then published the newsletter containing all these things. So from this:

Figure 100 - Page editor using keywords to load different types of post

We get this:

Figure 101 - Finished newsletter