Thursday, May 28, 2009

Text to Speech

Update: there's a series on Text-to-Speech eLearning:

More coming on that.

As part of the eLearning Tour, Christopher von Koschembahr demonstrated a podcast where he conducted an interview. The interviewer was a female voice asking questions. She would ask a question and then he would respond. What was interesting is that the interviewer was produced using Text to Speech. You can listen to the result at the 12:23 mark of the first video on the eLearning Tour page.

The effect was interesting. For short bursts of text, the computer generated voice was okay. Thus, it works well with the interview structure. Of course, you quickly know that he's written the questions. Still it works for me. In general, I think the response during the session was positive.

Based on the comments during the session, a few questions came up:

  • Does Text to Speech work for longer form content?
  • Captivate 4 contains built in Text to Speech. Are people using that much? For what kinds of content?

I did a couple of tests with a few of the free Text to Speech services that I found to see what something that's a big longer would sound like. I tried YAKiToMe!, Feed2Podcast, SpokenText, and ReadTheWords. None of them produced something that I could listen to for any length of time. My conclusion -

Text to Speech works for short bursts, not for longer amounts of content.

I'd be curious what anyone is doing out there with Text to Speech beyond the example that Chris discussed.

Anyone using it in their Captivate courses? How? How much text do you use?

Tuesday, May 26, 2009

eLearning - Social Media - Mobile Learning

Here's what you might have missed last week...

eLearning Learning Hot List - May 15, 2009 to May 22, 2009

Top Posts

The following are the top posts from featured sources based on social signals.

  1. The Challenge of Training the PlayStation Generation- The E-Learning Curve, May 15, 2009
  2. Metalearning- eLearning Technology, May 20, 2009
  3. Presentation: Blogs in Education- Don't Waste Your Time, May 22, 2009
  4. Aligning Learning Theory with Instructional Design- The E-Learning Curve, May 21, 2009
  5. List of System Variables in Cp4- Adobe Captivate Blog, May 15, 2009
  6. Presentation: Wikis in Education- Don't Waste Your Time, May 19, 2009
  7. How I use social media to learn- Adventures in Corporate Education, May 17, 2009
  8. Implementing New Learning Technology? Choose the Right Pilot Group- Kapp Notes, May 22, 2009
  9. Presentation: Social Bookmarking with Delicious- Don't Waste Your Time, May 15, 2009
  10. Discovering Instructional Design, Part 1- The E-Learning Curve, May 19, 2009
  11. Meeting icebreaker-How to get a group to acknowledge differences in perceptions.- Business Casual, May 16, 2009
  12. Discovering Instructional Design 3: A Systems Approach- The E-Learning Curve, May 22, 2009
  13. Shifting to Adobe eLearning suite – reuse existing Articulate Engage and Quizmaker content- Adobe Captivate Blog, May 19, 2009
  14. Social Learning Podcast Transcripts - Top Objections Webinar & Twitter Troubles- Engaged Learning, May 21, 2009
  15. MOBILE LEARNING - eLearning Tour Part 1 - Hosted by Corporate Learning Trends and Innovation- Discovery Through eLearning, May 21, 2009
  16. Learning, Extended Brain and Topic Hubs- eLearning Technology, May 18, 2009
  17. Road to Learning now featured on elearninglearning- Road to Learning, May 17, 2009
  18. Do not go quietly into the classroom- Don't Waste Your Time, May 17, 2009
  19. Should students be allowed Internet access in exams?- ThirdForce Blog, May 15, 2009
  20. US Army Using Interactive Videos- MinuteBio, May 15, 2009

Top Other Items

The following are the top other items based on social signals.

  1. A List Apart: Articles: In Defense of Eye Candy, May 16, 2009
  2. Facilitating Online | Centre for Educational Technology, May 19, 2009
  3. Are Your E-Learning Courses Pushed or Pulled?, May 19, 2009
  4. 25 Tools: A Toolbox for Learning Professionals 2009, May 19, 2009
  5. Learning 2.0 and Workplace Communities - 2009 - ASTD, May 18, 2009
  6. Sensemaking, PKM and networks, May 17, 2009
  7. The Mobile Learning Engine (MLE) for Moodle, May 18, 2009
  8. Learning Management Systems 2009 - 2009 - ASTD, May 21, 2009
  9. The Twitter Book, May 15, 2009
  10. Mashups for Learning - 2009 - ASTD, May 21, 2009
  11. Wolfram Alpha, May 21, 2009
  12. A landscape of influences, May 18, 2009
  13. E-learning: How to Perform Mime | Personal Cyber Botanica, May 21, 2009
  14. Educate - iPhone ipod Touch App for teachers, May 20, 2009
  15. 1,000+ Learning Professionals to follow on Twitter, May 16, 2009

Top Keywords

Financial Investment

Should learning organizations make a financial investment in new forms of learning?

A fantastic comment by Bill Brantley on my post Metalearning:

Before you start defining metalearning, you need definitions for:

  1. formal learning
  2. informal learning
  3. social learning
  4. collaborative learning
  5. personal learning

that are more than just marketing buzzwords.

What is the difference between these five concepts? What are the strengths and weaknesses with each? How does one know when they are practicing one form or another?

Before you start shutting down training departments, hiring Chief Learning Officers, and coining an umbrella term for different learning methods, you need to establish what you are actually talking about and why it is preferred over other methods. And you need to back this up with some empirical data.

I would love to have a discussion with him because I think he's missing the point about the importance of metalearning and metacognition and their implications on learning organizations. 

Important Challenge

That said, he's expressing a really important challenge.  Before a learning organization recommends to make a financial investment in any of these methods, they really want to know:

  • What is it?
  • What will it cost and what's the expected return?

When you look at various training methods such as classroom instruction, virtual classroom, courseware, online reference, performance support tools, they all have fairly well understood size, shape, characteristics.  There's enough body of knowledge, history and expectation that you can safely propose financial investment by a learning organization in these methods.  Yes, your budget is being cut, but it's way safer to propose on-going financial investment in a tried and true method than it is to propose shifting budget to new methods. 

In Corporate Training, I suggested what might happen if you shifted budget right now without having a solid backup as wonderfully explained by Dilbert:

Dilbert.com

 

If we want to really change where learning organizations spend time and dollars, the key ingredient is to help get more concrete about these terms and to be concrete about financial investment proposals.

Not a Short Answer

I wish there were a set of business cases that we could point to that would exemplify what a VP Learning/CLO should be presenting to their executive team.  Why not?

It's partly that these kinds of solutions are highly fragmented.  Look at the breadth of Examples of eLearning 2.0.  Add to it all the investment that could go along with Tool Set and Work Literacy

What should a VP Learning / CLO present to the executive team?

 

Related:

Wednesday, May 20, 2009

Metalearning

I just got through reading Jay's post and article (with Clark Quinn) around Become a chief meta-learning officer – one of the hot list items from two weeks ago.  It's a great article, definitely worth a read.  It discusses the needed transition in focus of a CLO

The scope of the job of the CLO is mushrooming. CLOs will neither prosper nor even survive if they fail to take responsibility for the overall learning process within their organizations.

Your charter as Chief Meta-Learning Officer is to optimize learning throughout the organization, not just in the pockets that once belonged to HR. This takes a broader perspective than what you deal with day-to-day. You’ve got to rise above the noise to see the underlying patterns, and then optimize them.

The reality is that this equally applies to Learning Organizations and Learning Professionals.  The broader perspective he is talking about is to look beyond formal learning to informal learning, social learning, collaborative learning, and personal learning. 

What was particularly interesting about his article was the use of the term meta-learning.

Wikipedia defines meta-learning (in education) as:

The idea of metalearning was originally used by John Biggs (1985) to describe the state of ‘being aware of and taking control of one’s own learning’. you can define metalearning as an awareness and understanding of the phenomenon of learning itself as opposed to subject knowledge. Implicit in this definition is the learner’s perception of the learning context, which includes knowing what the expectations of the discipline are and, more narrowly, the demands of a given learning task. Within this context, metalearning depends on the learner’s conceptions of learning, epistemological beliefs, learning processes and academic skills, summarized here as a learning approach. A student who has a high level of metalearning awareness is able to assess the effectiveness of her/his learning approach and regulate it according to the demands of the learning task.

Jay's article is really using metalearning in a different way – take responsibility for learning across the organization.  Look at all the different ways learning can occur.  Close the training department.  Etc.

Still, when I read the title, I couldn't help but think that Chief MetaLearning Officer was particularly apt, especially when you take "metalearning" according to its definition above.  Metalearning is really about:

being aware and taking control of one's own learning

It is a critical element to success moving forward.  And it's exactly what I've been talking about over the past few years.  The only way to handle long tail learning is to focus on providing the tool set and personal learning and working skills (work literacy) that are central to concept work.  Where work and learning are not separate, metalearning is really the focus of performance improvement. 

In Learning, Extended Brain and Topic Hubs, the focus is really on a new process for learning.  Being aware and in control of the process is metalearning.  Nancy Devine in a comment suggested "schema building" which is similar to pattern identification.  But all of this is really about metalearning.

Way back in 2006, I wrote Improving Personal Learning - A Continuing Challenge for Learning Professionals, that also cited a CLO article on Implementing Learning How-to-Learn Strategies (which doesn't seem to exist anymore).  Three years later, we recognize the increasing importance, and the need for greater metalearning development opportunities and the role of learning professionals and learning organizations in this.

Is metalearning a good term to encapsulate what we are talking about?

Tuesday, May 19, 2009

Twitter TechSmith LinkedIn Learning Strategy

Hot List - May 8, 2009 to May 15, 2009

Once again, used eLearning Learning to generate a list of the best from last week.

The following are the top posts from featured sources based on social signals.

  1. The Truth About Twitter- Social Enterprise Blog, May 11, 2009
  2. Twitter Tips: for Teachers & Educators- Don't Waste Your Time, May 9, 2009
  3. Twitter and Webinars- eLearning Technology, May 14, 2009
  4. Developing a PLE Using Web 2.0 Tools- Don't Waste Your Time, May 10, 2009
  5. Informal Learning Technology- eLearning Technology, May 11, 2009
  6. Presentation: Social Bookmarking with Delicious- Don't Waste Your Time, May 15, 2009
  7. The Ten Commandments of eLearning- Upside Learning Blog, May 8, 2009
  8. Online Coaching- eLearning Technology, May 13, 2009
  9. Audio in eLearning: Cultural Differences?- Learning Visions, May 12, 2009
  10. Presentation: Twitter in Education- Don't Waste Your Time, May 12, 2009
  11. Overcoming Objections to Social Learning - One Week at at Time- Engaged Learning, May 8, 2009
  12. The Challenge of Training the PlayStation Generation- The E-Learning Curve, May 15, 2009
  13. Presentation: Social Bookmarking with Delicious- Don't Waste Your Time, May 15, 2009
  14. Designing engaging e-learning- Clive on Learning, May 11, 2009
  15. Lies, damned lies, and Wikipedia…- ThirdForce Blog, May 8, 2009
  16. Hashtags in Twitter and walls, fountains, ways to keep everyone's remarks in the picture- Ignatia Webs, May 15, 2009
  17. Social learning adoption?- Road to Learning, May 8, 2009
  18. Learning, Models and Other Tricks- Blogger in Middle-earth, May 9, 2009

The following are the top other items based on social signals.

  1. The Eight Classic e-Learning publications? | Tony Bates, May 8, 2009
  2. The End in Mind " A Post-LMS Manifesto, May 8, 2009
  3. Does technology change the nature of knowledge? | Tony Bates, May 8, 2009
  4. Engage Your Learners By Mimicking the Real World, May 12, 2009
  5. A closer look at using a social media platform ..., May 10, 2009
  6. The Twitter Book, May 15, 2009
  7. The Learning Age, May 14, 2009
  8. All information is suspect, May 12, 2009
  9. How to Get the Most Out of a Conference, May 7, 2009
  10. Designing Authentic Learning Tasks, May 11, 2009
  11. Jing - The Missing Manual, May 10, 2009
  12. Coaching informal learning, May 9, 2009
  13. Adding value to information, May 12, 2009

Hot Keywords for the Week

Thursday, May 14, 2009

Twitter and Webinars

The last few online webinars I've attended there's been an interesting issue.  Many of the participants are Twitter users who are becoming used to chatting via twitter.  So, both the webinar tool (Elluminate, WebEx, Adobe Connect, etc.) have a chat stream happening and there's one going on via twitter.

There's some good information via Speaking Pro Central on twitter from a speaker's perspective including these great posts:

There are lots of good suggestions in these, but they don't really address the issue of how to present when the audience may be splitting their chat between the webinar tool and twitter.

In the meeting that I ran, I suggested that we would prefer that everyone uses the chat inside the webinar tool.

Advantages of the Webinar Tool

  • No switching applications
  • Everyone (not just twitter users) can see the chat
  • Avoids annoying people on twitter who don't want to see a flood of chat messages

Advantages of Twitter

  • Viral effect – may draw additional people into the webinar
  • Possibly engage with people who are not in the webinar
  • Accessible outside the webinar tool (searchable, etc.)
  • Comfort with the tool
  • Easy ability to follow or at least find out more about people who are interesting during the chat

I was planning to put in the advantages of both, but I can only think of the glaring disadvantage – you have to jump around a lot to see what's happening in both places.  You likely will have some people cut off from parts of the conversation.

Twitter Chat Annoying?

Actually, before I go any farther, I've got to ask:

Doesn't anyone else find the use of Twitter as a true Chat Channel a bit annoying? 

I expect twitter users to have a few updates and I certainly like when they tweet while they are listening to a presentation.  You often get some interesting nuggets.  But I don't like it when they start truly chatting because it turns into noise very quickly with messages that have no context.  Yes, you can go through the effort of filtering them out for a while, but that's annoying to have to do as well.

The annoyance level is not enough for me to say – don't use it for chat.  And I guess I'm going to have to come up with better strategies to handle this kind of bursty usage.

Short Term Right Answer?

So, what do you do when you are holding a webinar that has chat and where many of the users are twitter users as well?

For right now, I've been asking people to chat using the webinar tool.  And most twitter users are fine with that as the mode of operation.  Of course, they may tweet something that they believe people outside of the session would find interesting.

But I'm not sure if this is the right answer.  Thoughts?

Long Term Right Answers

Has anyone else noticed that webinar software vendors seemed to have stagnated their feature sets?  I've predicted for a couple of years that they would provide a 2.5D environment to give more presence to meetings.  Nothing.  Pretty much they are all looking alike.  Well, Mr. Webinar vendor, here's your chance to jump out in front of your competitors.

Webinar vendors should help us address this by providing outbound and inbound to twitter.  For outbound, we should be able to choose any or all of our messages or any links we see or anything like that during the presentation to be able to be pushed out into Twitter.  These will be associated with a hashtag for the event.  On the inbound side, the webinar chat should monitor the hashtag on twitter and pulls in any chats from outside right into the stream and associates it with the webinar participant or as an "outside" twitter person.  Basically, this uses twitter as an extended channel for the chat, but keeps a single view in the tool of the conversation. 

Which brings us to the other aspect of this – profiles in webinar tools.  I recently complained that even after all these years, tools like Elluminate and WebEx still didn't provide the ability to have people put their pictures and other information on their profiles so you could find out more about the users.  Come to find out they do, BUT its buried.  One of the big advantages of Twitter as a chat channel is that it extends seamlessly out to the rest of the social grid so that you can find out who this person is, likely see their blog, their LinkedIn profile, etc.  Why are profiles so buried in these webinar tools?  Yes, it takes a little bit to get set up, but with OpenId you could probably make this very easy for users.  Obviously to make the twitter thing work, users would have to provide their twitter credentials, so having this kind of profile information becomes more important.

Webinar Vendors should make it easier for us to go from your tool to find out more information about the participants including after the event and possibly link up with them via twitter, linkedin or other sites.  Heck, if you want to get fancy, you could probably take the online profiles of the webinar participants and show us all sorts of interesting things based on common elements of their profiles such as pages commonly linked via social bookmarking, common other groups/communities, etc. 

Think outside the walls of your tool – the folks in that session exist beyond the webinar.

Oh, and Webinar Vendors, you might want to look at Ping.fm as a model for helping entrants to update their status that they are attending the webinar at the start.  The hashtag and title might be there automatically.  Helps all of us get the viral aspects going. 

Not sure if any of the vendors will read any of this and I'm not sure if there's a lot we can do in the short run to work around the functional deficiencies of these tools.

Again, I'd love thoughts on this.

Wednesday, May 13, 2009

Online Coaching

Catherine Lombardozzi recent post Coaching informal learning sparked something for me around online coaching opportunities.  In prior posts she identifies the following elements as being needed for informal learning strategy to be effective in the workplace:

  • Motivation for learning.
  • A culture that provides access to other people who support learning in a wide variety of ways
  • Easy access to materials that support learning
  • Skills in utilizing electronic tools to manage learning.

In this post, she adds to the list that a "learning coach" is also needed:

Many learners will benefit from having someone to coach them through identifying their learning needs, sorting out the options for learning and development, and processing what they learn when they follow any of the available paths for learning.

I completely agree with Catherine that we need to be thinking about how learners can get support for dealing with the complexities of informal learning.  There's a new level of responsibility on learners.  Before I get into coaching, let me step back and look at the larger picture.

Learning Ignition Points

I believe that Catherine is describing the needs around informal learning when it's being used by workplace learning organizations to support learning – but learning somewhat distinct from work.  In other words, possibly supporting ongoing learning to build management skills for a group of new managers.  It's somewhat event oriented.  And likely defined initially in terms of a set of learning objectives.

I most often think about informal learning in the context of work objectives.  As I've said many times, for concept workers work and learning are inseparable. For a knowledge worker, generally its something like the start of a new project or a new kind of situation that sparks the need for learning.  We might call these:

Learning Ignition Points

But these ignition points are generally not coming from the learning organization.  They originate based on the work itself. 

Role of Learning Organizations

The learning organization's role is to provide support to these workers/learners as they hit learning ignition points.  In some cases, learning organizations may be aware of upcoming learning needs and be able to be out in front.  For example, an organization may make a strategic decision to go after a new market.  The learning organization can get out in front by looking to provide access to information about this market and talk with key stakeholders about possible changes that they may need to support to help accomplish this transition.

The vast majority of learning ignition points occur for individuals or teams based on their specific work.  For example, prior to going after a new market, possibly a single individual or a work team is looking at various new markets and trying to figure out what might make sense for the organization.  This is why most learning is long tail learning.

The role of learning organizations around long tail learning is to provide tools, support, skill building so that learners can self-serve their learning.

Really that's what Catherine is talking about as well.  But the difference in the perspective of work needs vs. learning needs is important.  Learning organizations cannot think about this in terms of "creating informal learning events" … they need to think about …

How do I support concept workers when they hit learning ignition points?

Coaching

There's actually quite a bit that learning organizations can and should do to support concept workers at learning ignition points, but I want to focus back on the idea that Catherine raises: coaching.

When a knowledge worker hits a learning ignition point, they may or may not think of this as "I need to learn."  Rather they may think about "getting up to speed" or "finding out about" or …

In a prior post, I talked about Dave Pollard's experience with knowledge management (and it's a conversation I've had directly with Dave):

So my conclusion this time around was that the centralized stuff we spent so much time and money maintaining was simply not very useful to most practitioners. The practitioners I talked to about PPI [Personal Productivity Improvement] said they would love to participate in PPI coaching, provided it was focused on the content on their own desktops and hard drives, and not the stuff in the central repositories.

In fact, much of what Dave talks about is his transition for KM as centralized solutions, to going out to help support and coach.  This assumes that the learning organization or someone has spent time to get to workers ahead of their learning ignition point to provide them with support and to be in position to coach.

The reality is probably more of what Catherine talks about when she asks the important question: Who are these coaches? 

For workplace learning, this is a critical role of the line manager.   A good manager develops people, and there is no more powerful way to do that than to be an encouraging and demanding learning coach.  If managers take on this role, the question about how to monitor and evaluate informal learning dissolves; managers will be intimately involved in knowing what, how, when and to what degree their direct reports are learning.

Likely for many workers when they hit a learning ignition point, they do go to their line manager to ask for thoughts, help, etc.   And I wonder about the skills that most line managers possess around learning coaching.

Online Coaching

While the line manager and possibly the learning organization may be providing some of the coaching, my guess is that a lot of the coaching comes from quite varied sources that I would roughly say are peers inside and outside the organization.

When I look at my own behavior, I take any new learning need and consider whether it's something I can likely just find through search, or if it's more complex, then I quickly move for learning need to the key question:

Who do I know who can help me figure out how to learn about this?

In many cases, the answer is that I'm not sure who I should talk to about it.  In which case, the first bit of effort is what I refer to as Conversation Seeking in the post: Networks and Learning Communities.  What I said at that time still holds that most often I find myself using:

  • LinkedIn
  • Various learning communities
  • My Blog
  • Twitter

And the question I'm asking is always of the form:

Here's what I know, but I'm trying to find out X, how should I go after that?

Recent examples for me are …

  • Business side of professional speaking
  • Aggregation technology
  • Research on categorization / types of eLearning
  • New Way of Learning

In fact, if you run down my blog posts you could consider about half of these to be part of this.  Likely about 10% of my twitter posts are seeking information.  Most of my online coaching seeking activity you don't see because it's one-to-one via LinkedIn.

Again, you can go back to Networks and Learning Communities to see quite a bit about all of this.

What's interesting to me is how much of what I personally do when I hit a learning ignition point is really seeking and getting online coaching.  But it shouldn't just be me.  It should be all knowledge workers: 

Knowledge workers should have the tools and skills to
utilize online coaching as they hit learning ignition points.

Upfront vs. Time-of-Need Coaching

One thing I've realized as I'm writing this is that there is likely two times when a learning organization might be involved in coaching.  One point is prior to a specific learning need in order to help the knowledge worker be ready when they do hit the specific need.  This is really what Dave Pollard is talking about.  Go through the organization and provide tools, support and skill building.  It's worth looking at some of his additional specific suggestions in Knowledge Management: Finding Quick Wins and Long Term ValueHis 6 Quick Wins are:

  1. Make it easy for your people to identify and connect with subject matter experts.
  2. Help people manage the content and organization of their desktop.
  3. Help people identify and use the most appropriate communication tool.
  4. Make it easy for people to publish their knowledge and subscribe to the information they want.
  5. Create a facility for just-in-time canvassing for information.
  6. Teach people how to do research, not just search.

The other point where learning organizations might be involved is at time-of-need.  A learning ignition point has occurred.  Are you in position to try to help them.  This is a bit more like the role of a librarian.  It's specific help on how the knowledge worker can effective learn about this new area.

Actually this raises an interesting question around the intersection of corporate librarians 2.0 and learning organizations 2.0.  Any specific pointers on that?

What's Next

In looking at this, it's hard to believe that time-of-need coaching is going to happen if we've not done the upfront coaching.  So, the reality is that we need to be looking at Workplace Learning Professionals Next Job - Management Consultant.  Really that's what we are talking about here.

I would appreciate any thoughts on this?

Tuesday, May 12, 2009

Desire2Learn Mobile FLV - eLearning from Last Week

Here what eLearning Learning thinks is the most interesting stuff from last week. Actually, it's what all of you seemed to think was the most interesting stuff from last week.

Top Posts

The following are the top posts from featured sources based on social signals.

  1. New Way of Learning- eLearning Technology, May 4, 2009
  2. The Ten Commandments of eLearning- Upside Learning Blog, May 8, 2009
  3. Avoiding the Virtual Ghost Town- Kapp Notes, May 6, 2009
  4. Learning Outcomes- eLearning Technology, May 6, 2009
  5. ASTD / ISPI Event- Social Enterprise Blog, May 8, 2009
  6. eLearning Tour - May 21 - Free- eLearning Technology, May 5, 2009
  7. Brain rule #6- Clive on Learning, May 8, 2009
  8. Improved Learning or Business Benefits- eLearning Technology, May 4, 2009
  9. Lies, damned lies, and Wikipedia…- ThirdForce Blog, May 8, 2009
  10. Profile Photos- eLearning Technology, May 7, 2009
  11. The next Web of open, linked data (Semantic Web)- Don't Waste Your Time, May 6, 2009
  12. Describing What You Do: Instructional Design- Learning Visions, May 6, 2009
  13. iPhone Learning Links- Kapp Notes, May 5, 2009
  14. How small is too small for educational technologies to be meaningful?- Electronic Papyrus, May 3, 2009

Top Other Items

The following are the top other items based on social signals.

  1. 9 Free Tools That Help Me Build Better E-Learning, May 5, 2009
  2. Control and Community: A Case Study of Enterprise Wiki Usage, May 4, 2009
  3. Using Elgg as as Social Learning platform, May 2, 2009
  4. Learning with 'e's: e-Learning 3.0, May 4, 2009
  5. Learning 2.0, May 5, 2009
  6. Become a chief meta-learning officer, May 3, 2009
  7. The Future of eLearning is Social Learning, May 2, 2009
  8. Are You a Super Learner?, May 7, 2009
  9. Learning as a Network, May 7, 2009
  10. Is there no room for Informal Learning?, May 3, 2009
  11. How to Get the Most Out of a Conference, May 7, 2009
  12. New technology supporting informal learning, May 4, 2009
  13. Royalty-Free Music, May 2, 2009

Top Keywords

Email Address Bias

Paul Angileri who write the blog There Is No Chalk just left a comment on my recent post Profile Photos (see also Profile Photo):
I once had someone comment to me on a non-professional email address of mine. They said it seemed to imply something negative. This was the only person ever to have mentioned it to me, and for the life of me I still can't see the person's POV on it. There's certainly no profanity of vulgarity in the email address, yet someone had a mildly negative reaction to it and at the very least seemed to imply that the very structure of the email address would likely preclude that person from communicating with me.
Many of the recent comments really have helped to focus the discussion on the right issues - your brand, your audience and what they perceive. Subquark has convinced me that there are cases where a creative image and likely name make a lot of sense to help promote a creative business. It aligns with the brand. And if someone is not interested in interfacing with someone creative, that's not really the audience anyhow.

As long as we are talking about possible bias when someone encounters you online, let's discuss email addresses. Again, in professional networking, your email address should align with your brand, the perception that you want to convey with your audience and the cultural norms of where you are networking.

Names

It's safest to use your name. It's probably a bit of a risk to use any kind of non-name.

Again, the situation is a quick decision (less than a second) where you look at the person and decide if you will spend more time on this.

What do you get from each of the following?
  • snuggybear83
  • john.mcelhone
  • surferchamp
Again, there are likely cases where more creative names make sense. But make it a conscious decision.

Domain Bias?

The other thing I would suggest is a likely bias out there on domains.

Please help me out on this ... what do you think when you see the following domains?
  • aol.com
  • yahoo.com
  • gmail.com
  • johnmcelhone.com

Monday, May 11, 2009

Informal Learning Technology

There's a fantastic post by Stephen Downes - New Technology Supporting Informal Learning.  In his post, he really is looking primarily at the University of Manitoba's Connectivism Course that he designed and delivered with George Siemens to 2200 students in Fall 2008 and you can tell that he's busy thinking about the technology behind the Fall 2009 course.

So as you read this, keep in mind that Stephen was primarily talking about technology that can support a Formal Learning Event that heavily leverages Informal / Social Learning as it's primary mechanism.

It's definitely worth a read.  Somewhat random thoughts as I reviewed the post.

Training Modalities

Stephen tells us at the outset:

Online learning today is beginning to be dominated by developments in games, simulations and related technologies. (Akili, 2007)

I was just recently looking back at my post Training Methods and have been evaluating submissions as part of preparing for the upcoming eLearning Tour.  Both of these suggest that Stephen's use of the term "dominated by" is probably misleading.

Complexity

Stephen and I see similar kinds of complexities:

our best response to the variability and complexity of the subject matter along with the changing nature of the learner is to design systems that are decentralized, to push learning decisions down the hierarchy or out to the edges of the network.

Stephen goes through a great description of the course.  There's a lot to learn from that experience.

Certainly one of the things that jumps out at me was the same learning goals challenges that we faced in the work literacy course that we ran during a similar timeframe:

Because there were so many people contributing to the course, and because the content of the course actually shifted and varied according to participation and input into the course, it was necessary to emphasize to students that their role in the course was not to attempt to assimilate all course content. This was neither possible nor desirable. Rather, students were told that their role was to select and sample course content, pursuing areas of interest, reading related material from both within and outside the course, and then to contribute their unique perspective based on this reading.

For flow learners with flow learning goals, this likely worked quite well.  For directed learners, likely this was a bit disconcerting.

Still the fact that within the context of a formal learning event, they heavily use student activity aggregated together is a very interesting model.  In some ways, you could say that this might be an interesting model to experiment with as a formal/informal hybrid at the start of some new project.  A facilitated, get up to speed on a subject, kind of thing.

I'd be curious to hear thoughts on that kind of learning experience.

I also find it interesting to see the parallels to what I'm doing right now as I explore aspects of professional speaking.  I'm using aggregation and interactions with various bloggers on the topic to learn about it.  This is the same technology that is behind eLearning Learning and Communities and Networks Connection.  It allows multiple people to contribute to the aggregator and tries to filter and organize that content.  Stephen was dealing with that as part of the course and had to make daily sense of what was going on.  Possibly some of the social signals that we could use could have helped him, and I'll be curious how he automates more of that for the Fall 2009 class.

PLE Functions

Stephen's description of the main elements in the personal learning environment (PLE) was very interesting:

In the PLE project being undertaken by the National Research Council, the functionality of the PLE is depicted in four major stages: to aggregate, that is, to collect content from the individual's and other online content service providers, where aggregation includes elements of recommendation, data mining and automated metadata extraction ; to remix, or to organize content from several different sources in different ways, including through automated clustering; to repurpose, or edit, localize, or otherwise modify or create new content; and to feed forward, or send the content to subscribers and other web services, either via RSS syndication, email, Twitter, or other relevant services. (Downes, Theory of Learning Networks, 2004)

PLE functions:

  • aggregate
  • remix
  • repurpose
  • feed forward

It's interesting to see this perspective on the PLE where it is part of the Connectivism course and students are naturally motivated (or required) to share.  When I look at this from more the PWLE (Personal WORK and learning environment) perspective, I find that there are similar functions, but possibly a different mind set.

Looking back at my Tool Set series at the start of this year, there were very similar kinds of capabilities being described.  At first, when I looked at Stephen's list of functions, I felt like he was too content centric and had left out the people (see Networks and Learning Communities).  In many cases, I find myself searching for the right person as an answer to my learning / work need rather than finding the right content.  And I feel that Leveraging Networks is Key Skill and the most important Knowledge Worker Skill Gap.

However, as I read further in Stephen's description he said:

A network of PLEs is a learning network.

And certainly the effect of the network of PLEs is similar to the search for individuals that I describe.  People naturally congregate and interact around content that they've found interesting.  This is an issue that I've found fascinating about networks – content-centric vs. people-centric.  If you think of Flickr, Wikipedia, Delicious – these all are content-centric networks.  The content is the central element but there's lots of social aspects and interaction.  Whereas when you look at LinkedIn (leaving aside Answers and Groups), it's a people-centric network.  LinkedIn certainly is adding all kinds of content-centric connection opportunities with Answers and discussions via Groups. 

Thanks Stephen for a great post and a lot of insights into the technology side of supporting informal learning.

Thursday, May 7, 2009

Profile Photos

I received a response that I didn't expect on my post Profile Photo.

In that post, I suggested that you should always add a profile photo:

Don't join that new Ning group without attaching your photo. Add it to your Elluminate and WebEx profiles.

Because profile photos help me:

  • Believe that you are a real person and begin to connect with you.
  • Remember you.
  • Believe you are serious.

I also suggested that for business networking:

  • Don't use anything other that photorealistic photos.
  • Have a reasonably complete LinkedIn profile.

The comments I received basically told me that I'm making unfair, snap judgments and I'm showing my bias. And I want to thank everyone who commented and who called me on that bias.

You are quite right that I'm biased. But let's set the context here –

  • I don't already know you or your name.
  • I'm running into you on a business networking site.
  • All I have is often your Photo, Name and sometimes a Company and Title.
  • I have limited time.
  • My goals are generally directed learning goals or maybe I'm thinking about issues with my business or my clients.
  • I need to quickly decide if I'm going to spend time with you and/or your content. Will this be a smart use of time?

That sounds horrible. I'm going to use that little bit of information to decide if I will go look you up on LinkedIn, or send you a message about something, or otherwise interact. But that's the reality of what happens.

And profile photos are part of the picture.

This is not new. It's happened for years at business mixers. People need to control their time as part of in-person networking. They also make snap judgments at business mixers. And they have the person, their name, possibly title and company. They make quick judgments. They choose to speak to your or not.

I'm not saying this is fair or right … I'm saying this micro-decision is going on all the time and you are not likely to change it (although you've got me to think about my own bias). I think you ignore it at your peril.

Now consider the impression left by the images that I've pulled down from various business networking sites shown below. And consider that people are going to make snap judgments on whether they will spend time.

image imageimageimageimageimageimageimage

Dating and Profile Photos

Because I was involved in eHarmony at the start (see Matching Algorithm), I may be especially sensitive to this subject. eHarmony was (and is) quite different from dating sites in that the true vision was to create successful (happy and long) marriages. And they have been successful doing that. For a long time eHarmony resisted showing pictures because of the belief that this skews someone's impression of the individual, and you may choose someone other than your soul mate because of a profile photo. Users demanded a profile photo.

This is not much of a surprise. Here's what was found on Yahoo personals:

When our researchers looked at personal photo statistics, here's what they've found:

  • A profile with one personal photo receives five times as many replies as a profile without a personal photo.
  • A profile with three photos receives seven times as many replies as a profile without a personal photo.
  • A profile with five photos receives nine times as many replies as a profile without a photo.

Granted, eHarmony has a very different mission and purpose from a personals site like Yahoo, but profile photos turned out to be important there as well.

And what does this tell us – profile photos are important.

Profile Photos Other Opinions and Advice

Seth Godin tells us that profile photos are important in business as well:

If it's important enough for you to spend your time finding and connecting with new people online, it's important enough to get the first impression right.

If you use any online social network tool, the single most important first impression you make is with the 3600 to 5000 pixels you get for your tiny picture.

Seth's number 6 advice backs me up on conceptual photos/images:

6. Conceptual photos (your foot, a monkey wearing glasses) may give us insight into the real you, but perhaps you could save that insight for the second impression.

There's definitely belief in the book business (and some debate) that having the right picture is very important for book sales.

Antonia Hodgson, the editor-in-chief at the publisher Little, Brown, it’s more important than ever, “The author photo is now just the beginning of a process of getting to know the author.”

And I would claim profile photos are the starting point in building a relationship online.

A couple of other interesting links that I found on this topic:

Closing Thought

One closing thought – you may look good in a bathing suit, but you don't wear it to a business mixer. Or at least I think I would remember you if you did. Instead, you look the part. Profile photos are part of your image, your brand. Look the part.

Wednesday, May 6, 2009

Learning Outcomes

My post on learning goals has some great feedback via comments and a couple of blog posts. The focus of the post is really about the design of informal / social learning experiences and my belief that they need to have context for what I call Directed Learners. Particularly, there still needs to be Learning Outcomes defined. But please don't read Learning Outcomes to mean Learning Objectives. They are distinct and while I'm not claiming to have a great definition here, I think you can get the flavor from this post.

Business Outcomes and Learning Outcomes

This relates to my recent post about Improved Learning or Business Benefits. However, the context of that post is more about defining the overall benefits / positioning. Still in my learning goals post I said:

"Unlike formal learning, informal learning is generally not going to ensure that specific knowledge will be transferred. Instead, people will learn what they need in order to accomplish the ultimate objectives. We aren't sure what they will learn."

and Jay Cross responds:

In a business context, isn't learning enough to accomplish the objective sufficient?

At a more fundamental level, the massive swing from the industrial age to the network era is accompanied by pervasive uncertainty. Twenty years ago, the business world seemed predictable; corporations wrote five-year plans. Today, it's a world of surprises. We're all tied to the unpredictable interplay of complex adaptive systems. It's tough to assemble a viable learning to-do list when you don't know what tomorrow will bring.

Jay, I agree that it is extremely hard to define effective learning experiences in a chaotic world. It's even more challenging when you have to deal with Directed vs. Flow Learning Styles and with motivation and with questionable outcomes. That said, I don't think the answer is really (and I don't think Jay thinks this either) to ignore the challenges of working with Directed Learners. I actually think that informal learning experiences can be greatly improved by ensuring that you are dealing with learning outcomes or business outcomes.

Concrete Learning Outcomes

And to that end, great post by Michele Martin that responds to my learning goals post. She provides some context and then positions it in a question that I think is really great about specific learning experiences:

So the issue becomes, how do you define learning goals for social, informal learning in a way that provides context and makes sense for more concrete directed goal learners?

Michele has concerns that defining a learning outcome or business outcome or set of known learning objectives for the informal / social learning experience is a good start.

In my experience, this may be true of a directed learner like Tony, but he seems to be an exception. The people in my work who are most uncomfortable with informal, social learning are those who are also uncomfortable with something as ill-defined as a "business outcome" for their learning goals. They want very specific, concrete, actionable learning objectives AND they want a step-by-step process for getting there. In fact, I find that these types of learners have no patience for informal learning. To them, it's not learning at all. It's too messy and ill-conceived (in their minds).

Likely most of us can relate to this. Even if everyone in the room at the end of the session feels like they are better able to perform some task, but they all got there a different way, there will be some members of the audience who aren't sure if they saw the right stuff, missed something … it's just that uncomfortable feeling of not being sure if you really got it.

Michele may be right that:

I'm not sure that it's possible to provide enough of a structured goal orientation to informal learning to totally satisfy most directed goal learners.

That's probably true – but I'll guess that people are likely somewhere along the spectrum and while the extreme ends of directed learners may not be happy with an outcome that achieves the purpose but has an uncomfortable path, I believe that you can greatly increase the comfort of directed learners with the right kinds of context setting.

Michele pretty much lands right there:

In designing informal learning activities, we may need to get better at helping directed and flow learning people forge a learning path for themselves to navigate the social learning space. But that's a good thing--because then we're also helping people develop the skills they need to learn from work itself.

Well said!

Social Learning Design Revisited

What's also interesting about this is that Michele, Harold Jarche and I went through the whole design of the Work Literacy Ning learning event and had several discussions about how the nature of the social learning environment might feel uncomfortable to many of the participants. But until now, I didn't really have the words to describe the issue.

Drilling down a bit on this, we did start out by collectively defining some high level goals. For example:

The goals of the event are to:
* Introduce you to new tools and methods for work and learning
* Discuss implications of these tools for learning professionals
*Prepare you to participate in DevLearn in new ways as an attendee or as a spectator.

And then for a particular week such as the one on Social Networking, there would be a list of more specific goals:

different topical areas to explore:
**The Tools--LinkedIn, Facebook and Ning are the primary social networks being used by learning professionals.
** How people use these networks for personal professional development
**How people use social networks to facilitate learning
**Issues and concerns related to privacy, managing multiple networks, and the value of these networks in terms of the time commitments involved.

But we knew that no one could possibly "explore" it all … so we also said:

Also--DO NOT FEEL YOU HAVE TO DO EVERYTHING! (Yes, I just shouted that). We have given you many different ways to dip in and out of these topics and by no means are you expected to do everything. You don't even have to read every forum on social networking or do all the activitities related to any particular topic. We have a few hundred people here and we wanted to give people a variety of ways to explore different aspects of social networking. This module is not linear--it's meant to give you an introduction and some avenues into exploring and discussing the tools. As Harold Jarche has said, "there are no gold stars." This is for your learning, so feel free to pursue it in the ways that work best for you.

Here is where we start to have a bit of a challenge because if I don't have to do everything and there's too much to explore, how do I know when I'm done?

In looking back, a lot of our focus was in describing the process and the systems that we would use to allow discussion. And we focused on topics to explore. I think we could have spent a bit more time defining some specific learning outcomes or at least have made some parts have higher importance. For example, if the point of the experience is to help learning professionals understand implications of these tools for themselves, then we might make an assignment to: examine how these tools fit into your personal work and learning environment. Or the assignment could be to write a list of five suggestions that they can present to their manager or someone in the business. Or ???

Basically, I would work with Michele and Harold to define a business / learning outcome that is close to what you would really want to achieve for the business.

Of course, the flow learning types might not like something to be mandatory or an assignment or have higher precedence or however you want to say it. They may have to put up with me trying to put some directed learning structure on top of their flow learning event. Having someone write a discussion item, blog post, or whatever that is the ultimate outcome that we are going for is really something that I would push for in hindsight.

I have no idea what Michele, Harold and I would have finally done as an outcome of this discussion. But I'm glad to have the context to discuss it now.

Tuesday, May 5, 2009

eLearning Tour

On May 21, 2009, I moderated an eLearning Tour in association with Learn Trends. It was designed to take you on a quick tour of different examples of eLearning Solutions happening out there. The tour guides and their topics:
  • Judy Brown - Mobile Learning Solutions
  • Bob Mosher - Performance Support Tools
  • Karl Kapp - Games and Simulations
  • Tony Karrer - Self-Paced and Other eLearning Solutions
Below are the videos from the sessions:

Mobile Learning


Performance Support


Simulations and Games


Asynchronous eLearning



Mobile Learning
Judy Brown, Tour Guide

For the mobile learning section of the tour, we will be talking about the opportunities and benefits in using devices our learners are already carrying for various types of learning. The topics / presenters will be:

Overview/Introduction – a quick scan of the current and future environment
  • Judy Brown, Mobile Learning Strategic Advisor, mLearnopedia

Innovative Podcasting – Introduction of research and an example of improved audio communications
  • Christopher von Koschembahr, a pioneer in mobile learning previously with IBM

Mobile Lessons Learned – Overview of experiences in the first year of use of iPhones and iPods for all the learners at Abilene Christian University
  • William Rankin, Associate Professor of English & Director of Mobile Learning Research: iPhone
  • George Saltsman, Director of the Adams Center for Teaching & Learning and Director of Educational Technology
Performance Support

Bob Mosher, Tour Guide

In the Performance Support tour we will be looking at technologies and approaches which bring learning directly to the learner’s “moment of need”. Performance Support offers a unique opportunity to go beyond learning and acquiring knowledge directly in the context of doing work. 4 case studies will be shared. The topics / presenters will be:

Overview/Introduction – a overview of the PS learning landscape and some overall design principles which need to be considered

  • Bob Mosher, Chief Learning and Strategy Evangelist, LearningGuide Solutions US

Electron Performance Support integration – 2 examples of highly guided and integrated Performance Support environment for IT applications and the DoD.

  • Dan Peay, Transcensus

Performance support for Processed based learning – Two more examples of PS used for helping support Medical Devices and On-boarding.

  • Bob Mosher, Learning Guide

Simulations and Games
Karl Kapp, Tour Guide

In this part of the tour, we'll explore how organizations are using games and simulations to provide engaging, effective training to their employees. You will see how:
  • Mini games
  • Game Engines
  • Machine Simulations
  • Soft Skills Simulation
are being leveraged to transfer knowledge within organizations.

Speakers during this section will be:

Adam Nelson who currently leads the learning group at Linden Lab, the creators of Second Life, designing and implementing next-generation adaptive learning environments in enterprises of all sizes. Prior to joining Linden Lab, Adam spent 10 years at Ninth House, a leading leadership and behavioral change company.

Phil Sueper - Simulation Designer for The Performance Development Group. Over the past nine years, he's designed more than 200 performance simulations. He will present a series of simulations with very brief summaries of the unique approaches to learning included in each

Self-Paced, Asynchronous eLearning
Tony Karrer, Tour Guide

In this part of the tour, we will quickly go through a series of demonstrations of different types of asynchronous, self-paced eLearning that comes directly from the submissions.

The topics / presenters will be:

Really Rapid eLearning - A rapid process, integrating SMEs to produce software overviews.
Ben Duffy, eLearning Program Manager, Fairchild Semiconductor

Narrative eLearning - Teaches about Pandemic Influenza Preparedness at Work and at Home through conversations at work and at home, and newscasts - incorporating audio and bandwidth constrained “video.”
Kim Koehler, Learning Center, Park Nicollet Health Services

Scenario eLearning - Introduces AmeriCorps VISTA to new members through an exploratory environment with interactive scenarios.
Angela Nicholas, Instructional Designer, Northwest Regional Educational Laboratory

Non-linear, Clue/Puzzle eLearning -
Introduces banking staff to a banking service and customer enrollment through a film noir setting where learners play a private detective search for clues.
Jeffery Goldman, e-Learning Designer, Provident Bank

Social eLearning - A reusable product that invites conference audiences into a conversation with keynote presenters before and after the event.
Douglas Flather, Vice President, Product Development, Washington Speakers Bureau Multimedia

More on our tour guides:

Karl Kapp

Karl is a consultant, speaker, scholar, and expert on the convergence of learning, technology and business operations. His background teaching e-learning classes, knowledge of adult learning theory, and experience training CEOs and front line staff provides him with a unique perspective on organizational learning. In 2007, he voted as one of TrainingIndustry.com's 2007 Top 20 Most Influential Training Professionals. Recently, he won the Innovative Excellence in Teaching, Learning and Technology. His experience with technology companies and high-tech initiatives provides him with insights into the future of technology. He shares those insights and perspectives through writing, consulting and coaching with clients in the field of learning and e-learning.

He is the author of "Gadgets, Games and Gizmos for Learning" and is working on a book about 3D Learning.

Judy Brown

Judy Brown is a Mobile Technology Analyst who has been involved in technology for learning for over 25 years and with mobile learning since 1996. She coordinates the mlearnopedia.com site. She moderates the Mobile Learning Content Community and is a fantastic speaker on all things mobile learning.

Bob Mosher

Bob has been an active and influential leader in the learning and training industry for over 23 years and is renowned worldwide for his pioneering role in e-learning and performance support. Bob joined LearningGuide from Microsoft, where he was Director of Learning Strategy and Evangelism. Before, Bob was the Executive Director of Education for Element K where he directed and influenced their learning model and products. He is an influential voice in the IT training industry, speaking at conferences and participating within industry associations such as CLO Magazine, CompTIA, ASTD, The E-learning Guild and The Masie Centre.

Tony Karrer

Dr. Tony Karrer is CEO/CTO of TechEmpower, a software, web and eLearning development firm based in Los Angeles, and is considered one of the top technologists in e-Learning. He has twenty years’ experience as a CTO. Dr. Karrer taught Computer Science for eleven years. He has been the acting CTO for several start-ups, most notably eHarmony. His work in social media, e-Learning and Performance Support has won awards and has led him into engagements at many Fortune 500 companies including Credit Suisse, Royal Bank of Canada, Citibank, Lexus, Microsoft, Nissan, Universal, IBM, Hewlett-Packard, Sun Microsystems, Fidelity Investments, Symbol Technologies and SHL Systemhouse. Dr. Karrer was valedictorian at Loyola Marymount University, attended the University of Southern California as a Tau Beta Pi fellow, one of the top 30 engineers in the nation, and received a M.S. and Ph.D. in Computer Science. He is a frequent speaker at industry and academic events.

Social Network Analysis - Twitter - Social Media - Best Stuff from Last Week

Here is the best stuff from last week via eLearning Learning.

Top Posts

The following are the top posts from featured sources based on social signals.

  1. Overcoming Top 10 Objections to Social Learning- Social Enterprise Blog, April 24, 2009
  2. Ten commandments of e-learning (content design)- Clive on Learning, April 30, 2009
  3. Twitter Learning- eLearning Technology, April 30, 2009
  4. Learning Goals- eLearning Technology, April 29, 2009
  5. An eight-step process using Post-it notes to gain meeting consensus.- Business Casual, April 25, 2009
  6. Fantastic File Converter- eLearning Acupuncture, April 24, 2009
  7. Call for eLearning Demonstrations- eLearning Technology, May 1, 2009
  8. When should you use Informal Learning?- Upside Learning Blog, April 27, 2009
  9. Online CEU Credits- eLearning Technology, April 27, 2009
  10. 508 Compliance, Even If You Do Not Need To- MinuteBio, April 30, 2009
  11. Diffusion of Innovations- Kapp Notes, April 30, 2009
  12. Gathering comments with Yahoo Pipes?- WISE Pedagogy, April 29, 2009
  13. eLearning mash-up follow-up: Articulate presenter and Flash 10 compatibility- Ignatia Webs, April 28, 2009
  14. Twitter Chat- Adventures in Corporate Education, April 24, 2009
  15. Steps for Designing a Virtual Learning World Experience- Kapp Notes, April 24, 2009

Top Other Items

The following are the top other items based on social signals.

  1. Enterprise: List of 40 Social Media Staff Guidelines, April 23, 2009
  2. NYTimes OpEd | End the University as We Know It, April 27, 2009
  3. The future of e-learning is social learning, April 25, 2009
  4. OpenGov: One big challenge? Or a thousand small hurdles : Tim's Blog, April 28, 2009
  5. Ten Commandments of eLearning Design, April 25, 2009
  6. 20 Ways to Evaluate Contributions to a Corporate Social Network | Dave Duarte, April 28, 2009
  7. Ignore This Post If You Don’t Care About Effective Learning, April 28, 2009
  8. Incredibly Dull: Social Architecture, April 28, 2009
  9. “Ba” for Management Development, April 24, 2009
  10. Emergent practices need practice, April 24, 2009
  11. Rough week for higher education, April 25, 2009
  12. The Business Case for Social Learning: Dealing with the "Capability Recession" at Lower Cost, April 30, 2009
  13. The intersection between work and learning, April 30, 2009
  14. Social Learning and Emerging Technology, April 23, 2009

Top Keywords