Wednesday, May 6, 2009

Learning Outcomes

My post on learning goals has some great feedback via comments and a couple of blog posts. The focus of the post is really about the design of informal / social learning experiences and my belief that they need to have context for what I call Directed Learners. Particularly, there still needs to be Learning Outcomes defined. But please don't read Learning Outcomes to mean Learning Objectives. They are distinct and while I'm not claiming to have a great definition here, I think you can get the flavor from this post.

Business Outcomes and Learning Outcomes

This relates to my recent post about Improved Learning or Business Benefits. However, the context of that post is more about defining the overall benefits / positioning. Still in my learning goals post I said:

"Unlike formal learning, informal learning is generally not going to ensure that specific knowledge will be transferred. Instead, people will learn what they need in order to accomplish the ultimate objectives. We aren't sure what they will learn."

and Jay Cross responds:

In a business context, isn't learning enough to accomplish the objective sufficient?

At a more fundamental level, the massive swing from the industrial age to the network era is accompanied by pervasive uncertainty. Twenty years ago, the business world seemed predictable; corporations wrote five-year plans. Today, it's a world of surprises. We're all tied to the unpredictable interplay of complex adaptive systems. It's tough to assemble a viable learning to-do list when you don't know what tomorrow will bring.

Jay, I agree that it is extremely hard to define effective learning experiences in a chaotic world. It's even more challenging when you have to deal with Directed vs. Flow Learning Styles and with motivation and with questionable outcomes. That said, I don't think the answer is really (and I don't think Jay thinks this either) to ignore the challenges of working with Directed Learners. I actually think that informal learning experiences can be greatly improved by ensuring that you are dealing with learning outcomes or business outcomes.

Concrete Learning Outcomes

And to that end, great post by Michele Martin that responds to my learning goals post. She provides some context and then positions it in a question that I think is really great about specific learning experiences:

So the issue becomes, how do you define learning goals for social, informal learning in a way that provides context and makes sense for more concrete directed goal learners?

Michele has concerns that defining a learning outcome or business outcome or set of known learning objectives for the informal / social learning experience is a good start.

In my experience, this may be true of a directed learner like Tony, but he seems to be an exception. The people in my work who are most uncomfortable with informal, social learning are those who are also uncomfortable with something as ill-defined as a "business outcome" for their learning goals. They want very specific, concrete, actionable learning objectives AND they want a step-by-step process for getting there. In fact, I find that these types of learners have no patience for informal learning. To them, it's not learning at all. It's too messy and ill-conceived (in their minds).

Likely most of us can relate to this. Even if everyone in the room at the end of the session feels like they are better able to perform some task, but they all got there a different way, there will be some members of the audience who aren't sure if they saw the right stuff, missed something … it's just that uncomfortable feeling of not being sure if you really got it.

Michele may be right that:

I'm not sure that it's possible to provide enough of a structured goal orientation to informal learning to totally satisfy most directed goal learners.

That's probably true – but I'll guess that people are likely somewhere along the spectrum and while the extreme ends of directed learners may not be happy with an outcome that achieves the purpose but has an uncomfortable path, I believe that you can greatly increase the comfort of directed learners with the right kinds of context setting.

Michele pretty much lands right there:

In designing informal learning activities, we may need to get better at helping directed and flow learning people forge a learning path for themselves to navigate the social learning space. But that's a good thing--because then we're also helping people develop the skills they need to learn from work itself.

Well said!

Social Learning Design Revisited

What's also interesting about this is that Michele, Harold Jarche and I went through the whole design of the Work Literacy Ning learning event and had several discussions about how the nature of the social learning environment might feel uncomfortable to many of the participants. But until now, I didn't really have the words to describe the issue.

Drilling down a bit on this, we did start out by collectively defining some high level goals. For example:

The goals of the event are to:
* Introduce you to new tools and methods for work and learning
* Discuss implications of these tools for learning professionals
*Prepare you to participate in DevLearn in new ways as an attendee or as a spectator.

And then for a particular week such as the one on Social Networking, there would be a list of more specific goals:

different topical areas to explore:
**The Tools--LinkedIn, Facebook and Ning are the primary social networks being used by learning professionals.
** How people use these networks for personal professional development
**How people use social networks to facilitate learning
**Issues and concerns related to privacy, managing multiple networks, and the value of these networks in terms of the time commitments involved.

But we knew that no one could possibly "explore" it all … so we also said:

Also--DO NOT FEEL YOU HAVE TO DO EVERYTHING! (Yes, I just shouted that). We have given you many different ways to dip in and out of these topics and by no means are you expected to do everything. You don't even have to read every forum on social networking or do all the activitities related to any particular topic. We have a few hundred people here and we wanted to give people a variety of ways to explore different aspects of social networking. This module is not linear--it's meant to give you an introduction and some avenues into exploring and discussing the tools. As Harold Jarche has said, "there are no gold stars." This is for your learning, so feel free to pursue it in the ways that work best for you.

Here is where we start to have a bit of a challenge because if I don't have to do everything and there's too much to explore, how do I know when I'm done?

In looking back, a lot of our focus was in describing the process and the systems that we would use to allow discussion. And we focused on topics to explore. I think we could have spent a bit more time defining some specific learning outcomes or at least have made some parts have higher importance. For example, if the point of the experience is to help learning professionals understand implications of these tools for themselves, then we might make an assignment to: examine how these tools fit into your personal work and learning environment. Or the assignment could be to write a list of five suggestions that they can present to their manager or someone in the business. Or ???

Basically, I would work with Michele and Harold to define a business / learning outcome that is close to what you would really want to achieve for the business.

Of course, the flow learning types might not like something to be mandatory or an assignment or have higher precedence or however you want to say it. They may have to put up with me trying to put some directed learning structure on top of their flow learning event. Having someone write a discussion item, blog post, or whatever that is the ultimate outcome that we are going for is really something that I would push for in hindsight.

I have no idea what Michele, Harold and I would have finally done as an outcome of this discussion. But I'm glad to have the context to discuss it now.

Tuesday, May 5, 2009

eLearning Tour

On May 21, 2009, I moderated an eLearning Tour in association with Learn Trends. It was designed to take you on a quick tour of different examples of eLearning Solutions happening out there. The tour guides and their topics:
  • Judy Brown - Mobile Learning Solutions
  • Bob Mosher - Performance Support Tools
  • Karl Kapp - Games and Simulations
  • Tony Karrer - Self-Paced and Other eLearning Solutions
Below are the videos from the sessions:

Mobile Learning


Performance Support


Simulations and Games


Asynchronous eLearning



Mobile Learning
Judy Brown, Tour Guide

For the mobile learning section of the tour, we will be talking about the opportunities and benefits in using devices our learners are already carrying for various types of learning. The topics / presenters will be:

Overview/Introduction – a quick scan of the current and future environment
  • Judy Brown, Mobile Learning Strategic Advisor, mLearnopedia

Innovative Podcasting – Introduction of research and an example of improved audio communications
  • Christopher von Koschembahr, a pioneer in mobile learning previously with IBM

Mobile Lessons Learned – Overview of experiences in the first year of use of iPhones and iPods for all the learners at Abilene Christian University
  • William Rankin, Associate Professor of English & Director of Mobile Learning Research: iPhone
  • George Saltsman, Director of the Adams Center for Teaching & Learning and Director of Educational Technology
Performance Support

Bob Mosher, Tour Guide

In the Performance Support tour we will be looking at technologies and approaches which bring learning directly to the learner’s “moment of need”. Performance Support offers a unique opportunity to go beyond learning and acquiring knowledge directly in the context of doing work. 4 case studies will be shared. The topics / presenters will be:

Overview/Introduction – a overview of the PS learning landscape and some overall design principles which need to be considered

  • Bob Mosher, Chief Learning and Strategy Evangelist, LearningGuide Solutions US

Electron Performance Support integration – 2 examples of highly guided and integrated Performance Support environment for IT applications and the DoD.

  • Dan Peay, Transcensus

Performance support for Processed based learning – Two more examples of PS used for helping support Medical Devices and On-boarding.

  • Bob Mosher, Learning Guide

Simulations and Games
Karl Kapp, Tour Guide

In this part of the tour, we'll explore how organizations are using games and simulations to provide engaging, effective training to their employees. You will see how:
  • Mini games
  • Game Engines
  • Machine Simulations
  • Soft Skills Simulation
are being leveraged to transfer knowledge within organizations.

Speakers during this section will be:

Adam Nelson who currently leads the learning group at Linden Lab, the creators of Second Life, designing and implementing next-generation adaptive learning environments in enterprises of all sizes. Prior to joining Linden Lab, Adam spent 10 years at Ninth House, a leading leadership and behavioral change company.

Phil Sueper - Simulation Designer for The Performance Development Group. Over the past nine years, he's designed more than 200 performance simulations. He will present a series of simulations with very brief summaries of the unique approaches to learning included in each

Self-Paced, Asynchronous eLearning
Tony Karrer, Tour Guide

In this part of the tour, we will quickly go through a series of demonstrations of different types of asynchronous, self-paced eLearning that comes directly from the submissions.

The topics / presenters will be:

Really Rapid eLearning - A rapid process, integrating SMEs to produce software overviews.
Ben Duffy, eLearning Program Manager, Fairchild Semiconductor

Narrative eLearning - Teaches about Pandemic Influenza Preparedness at Work and at Home through conversations at work and at home, and newscasts - incorporating audio and bandwidth constrained “video.”
Kim Koehler, Learning Center, Park Nicollet Health Services

Scenario eLearning - Introduces AmeriCorps VISTA to new members through an exploratory environment with interactive scenarios.
Angela Nicholas, Instructional Designer, Northwest Regional Educational Laboratory

Non-linear, Clue/Puzzle eLearning -
Introduces banking staff to a banking service and customer enrollment through a film noir setting where learners play a private detective search for clues.
Jeffery Goldman, e-Learning Designer, Provident Bank

Social eLearning - A reusable product that invites conference audiences into a conversation with keynote presenters before and after the event.
Douglas Flather, Vice President, Product Development, Washington Speakers Bureau Multimedia

More on our tour guides:

Karl Kapp

Karl is a consultant, speaker, scholar, and expert on the convergence of learning, technology and business operations. His background teaching e-learning classes, knowledge of adult learning theory, and experience training CEOs and front line staff provides him with a unique perspective on organizational learning. In 2007, he voted as one of TrainingIndustry.com's 2007 Top 20 Most Influential Training Professionals. Recently, he won the Innovative Excellence in Teaching, Learning and Technology. His experience with technology companies and high-tech initiatives provides him with insights into the future of technology. He shares those insights and perspectives through writing, consulting and coaching with clients in the field of learning and e-learning.

He is the author of "Gadgets, Games and Gizmos for Learning" and is working on a book about 3D Learning.

Judy Brown

Judy Brown is a Mobile Technology Analyst who has been involved in technology for learning for over 25 years and with mobile learning since 1996. She coordinates the mlearnopedia.com site. She moderates the Mobile Learning Content Community and is a fantastic speaker on all things mobile learning.

Bob Mosher

Bob has been an active and influential leader in the learning and training industry for over 23 years and is renowned worldwide for his pioneering role in e-learning and performance support. Bob joined LearningGuide from Microsoft, where he was Director of Learning Strategy and Evangelism. Before, Bob was the Executive Director of Education for Element K where he directed and influenced their learning model and products. He is an influential voice in the IT training industry, speaking at conferences and participating within industry associations such as CLO Magazine, CompTIA, ASTD, The E-learning Guild and The Masie Centre.

Tony Karrer

Dr. Tony Karrer is CEO/CTO of TechEmpower, a software, web and eLearning development firm based in Los Angeles, and is considered one of the top technologists in e-Learning. He has twenty years’ experience as a CTO. Dr. Karrer taught Computer Science for eleven years. He has been the acting CTO for several start-ups, most notably eHarmony. His work in social media, e-Learning and Performance Support has won awards and has led him into engagements at many Fortune 500 companies including Credit Suisse, Royal Bank of Canada, Citibank, Lexus, Microsoft, Nissan, Universal, IBM, Hewlett-Packard, Sun Microsystems, Fidelity Investments, Symbol Technologies and SHL Systemhouse. Dr. Karrer was valedictorian at Loyola Marymount University, attended the University of Southern California as a Tau Beta Pi fellow, one of the top 30 engineers in the nation, and received a M.S. and Ph.D. in Computer Science. He is a frequent speaker at industry and academic events.

Social Network Analysis - Twitter - Social Media - Best Stuff from Last Week

Here is the best stuff from last week via eLearning Learning.

Top Posts

The following are the top posts from featured sources based on social signals.

  1. Overcoming Top 10 Objections to Social Learning- Social Enterprise Blog, April 24, 2009
  2. Ten commandments of e-learning (content design)- Clive on Learning, April 30, 2009
  3. Twitter Learning- eLearning Technology, April 30, 2009
  4. Learning Goals- eLearning Technology, April 29, 2009
  5. An eight-step process using Post-it notes to gain meeting consensus.- Business Casual, April 25, 2009
  6. Fantastic File Converter- eLearning Acupuncture, April 24, 2009
  7. Call for eLearning Demonstrations- eLearning Technology, May 1, 2009
  8. When should you use Informal Learning?- Upside Learning Blog, April 27, 2009
  9. Online CEU Credits- eLearning Technology, April 27, 2009
  10. 508 Compliance, Even If You Do Not Need To- MinuteBio, April 30, 2009
  11. Diffusion of Innovations- Kapp Notes, April 30, 2009
  12. Gathering comments with Yahoo Pipes?- WISE Pedagogy, April 29, 2009
  13. eLearning mash-up follow-up: Articulate presenter and Flash 10 compatibility- Ignatia Webs, April 28, 2009
  14. Twitter Chat- Adventures in Corporate Education, April 24, 2009
  15. Steps for Designing a Virtual Learning World Experience- Kapp Notes, April 24, 2009

Top Other Items

The following are the top other items based on social signals.

  1. Enterprise: List of 40 Social Media Staff Guidelines, April 23, 2009
  2. NYTimes OpEd | End the University as We Know It, April 27, 2009
  3. The future of e-learning is social learning, April 25, 2009
  4. OpenGov: One big challenge? Or a thousand small hurdles : Tim's Blog, April 28, 2009
  5. Ten Commandments of eLearning Design, April 25, 2009
  6. 20 Ways to Evaluate Contributions to a Corporate Social Network | Dave Duarte, April 28, 2009
  7. Ignore This Post If You Don’t Care About Effective Learning, April 28, 2009
  8. Incredibly Dull: Social Architecture, April 28, 2009
  9. “Ba” for Management Development, April 24, 2009
  10. Emergent practices need practice, April 24, 2009
  11. Rough week for higher education, April 25, 2009
  12. The Business Case for Social Learning: Dealing with the "Capability Recession" at Lower Cost, April 30, 2009
  13. The intersection between work and learning, April 30, 2009
  14. Social Learning and Emerging Technology, April 23, 2009

Top Keywords

Profile Photo

I'm actively engaged in all kinds of sites that can be roughly described as social networking.  Through this, and because of my My LinkedIn Open Connection Approach where I basically treat LinkedIn like a really big business mixer, I have lots of opportunity to "meet" new people online.

My guess is that about 25% of the people I run into do not have a profile photo.  I'm writing this post to encourage you to:

  • Spend a little bit of time to create a reasonable profile photo
  • Attach it to every online profile when you create it

And when I say every, I mean every – don't join that new Ning group without attaching your photo.  Add it to your Elluminate and WebEx profiles.

It doesn't take a lot to add a profile photo, and it's fairly significant.  Why?

  1. It helps me believe that you are a real person.  That your request isn't from some faceless person out there in cyber space.  You are real.  If you walk up and talk to me at a big mixer, I will talk to you.  But if you are an unknown from cyberspace with no photo, it just doesn't feel as real.
  2. It helps me remember you.  I have a hard time associating names with faces.  A really hard time.  Your profile photo can be really helpful to me to connect the dots repeatedly.
  3. It shows me you are serious.  If I receive an inquiry or link/friend request from someone without a profile photo, I'm much more likely to ignore it.  It's a signal that you are just playing around.

A few other thoughts on this topic:

  • In business networking, using stylized photos or anything other than a normal photorealistic picture says – I'm playful and fun, but maybe not all that serious about all this stuff.  It's your choice, but I believe it hurts you on all three of the above items.
  • Have a reasonably complete LinkedIn profile and link to it everywhere.  When I meet someone online, I commonly go search for them on LinkedIn to see who they are and what they do.  If I can't find them or they have really limited information, that suggests they aren't as serious about online connections.  In my experience, it's not as effective to spend time with them online.  I'm not saying this is a rule, but there's a correlation in my experience.  And while I'm at it, if you have less than 200 connections, you probably aren't trying all that hard online either.

Monday, May 4, 2009

New Way of Learning

This month's Big Question - Social Grid Value and a comment by Ken Allan on my Learning Goals post really got me to thinking. Ken said:
I don't really think there's any new way of learning, but we may need to experiment with its delivery.
At first I nodded my head, but then I started to wonder...

Is there really no new way of learning?

This is similar to the question raised in Brain 2.0 and I'm not really sure where we landed in that discussion. At the time, I said:
I'm not claiming that the brain itself has changed, but instead what's changing is:
  • metacognition
  • metamemory
  • access to information
  • access to other people
  • access to smart systems
And I still feel we are vastly underestimating what is happening around all of this.
In fact, I still believe that this is THE challenge of the next 50 years plus. How do we accommodate the dramatic changes in the ways that humans interact with computers, information and each other? In 30 years we will be able to have implants that give us full, instant access to the web (no typing) including the social grid. Today, our access to all that information and other people is just a bit slower (we have to type). Sorry, I digress ...

So, back to the original question - new way of learning?

Part of the answer of whether there are new ways of learning is how we interpret the "way of learning." What is learning? Some definitions:
  • the cognitive process of acquiring skill or knowledge
  • refers to the acquisition, and transfer to long-term memory, of experience, Information, and Knowledge, which may subsequently be used for solving problems, making decisions, and creating new knowledge
If we are talking about brain function, I don't know enough to know whether there really is something different going on in how the brain works. So, if we interpret learning to be purely at that level, then an argument can be made that there's not anything different here because the brain still "learns" in the same way. Therefore the "way of learning" are the same. It's still some kind of physical activity within the brain. Once we are able to bypass visual / reading as our input mechanism from computers and have direct input into the brain that may be interpreted separate from those normal brain signals, then we can be assured that learning is different at this level. But I'm not sure that should be the hurdle we need to achieve in order to start to claim "new way of learning."

If I look at the result of learning which according to the above definitions:
  • skill or knowledge
  • subsequently be used for solving problems, making decisions, and creating new knowledge
I believe that so much is changed around this that it really has a profound impact on learning. We are already beginning down the 30 year path I'm describing above. Essentially we've begun to use computers as an extended brain. The social grid represents a Borg-like extended brain that gives us incredible access to expertise and problem solving. All of this changes:
  • what we learn
  • how we choose what needs to be committed to long term memory vs. electronic memory
  • what we need from a future look up standpoint
  • how we solve problems and make decisions
So while the physical level of learning may not be changing, the context has and is changing. It's changing enough that while it's technically accurate to claim there is no "new way of learning" - it feels misleading.

I believe the way I learn today is very different than the way I learned 10 years ago. And dramatically differently than I the way I learned 25 years ago.

Improved Learning or Business Benefits

Looking at the responses in Social Learning Designer to my services positioning template:

For _________ (buyers) we help in their desire to __________ (benefits)
by ________ (services) unlike others we _____________ (differentiation).

There were only three responses:

Jane Hart: I work with learning depts to help them create more participatory, collaborative approaches to learning - rather than just shoving content at people

Colleen Carmean: for learning orgs, we help in their desire to increase knowledge within the org by shaping systems that make info needed available to anyone at anytime. Unlike others, we do this by creating distributed spaces, places and tools for sharing, finding and creating knowledge.

John: For learning organizations we help build productive, skilled, thinking individuals and teams. Unlike others we do this in a collaborative, active, constructive manner.

This sparked a couple of quick thoughts …

Buyers / Audience – Learning Dept / Organizations

I'm guessing that each of these responses are from a vendor perspective as they define the audience the same and it's not the organization itself, it's the learning organization / department. 

Benefits – Learning / Knowledge

In my post Social Learning Measurement – Will Thalheimer points out

there are basically three reasons to measure learning:
1. To prove benefits
2. To improve the intervention
3. To improve learning.

And I think you could make an argument that our benefits could fall in any of the three camps:

  1. business benefits,
  2. better interventions,
  3. improved learning.

When I read each of the benefits above, I read them to be 3 - improved learning.  The closest to a real business benefit is "productive, skilled, thinking individuals" – but that's still a distance from what most business people would think of in terms of real business benefits.

This is really a critical positioning issue …

is our benefit improved learning or business benefits?

I realize there will be times for us to define ourselves as one or the other, but I hope that we seek to define ourselves in terms of business benefit.

In my post Social Learning Measurement – there are a host of things that we can measure around Social Learning (which was the focus of the post where these responses occurred).  These include things like:

  • Networking patterns
  • Learning efficiency
  • Learning outcomes
  • Contribution patterns 
  • Content usage patterns
  • Content quality
  • Idea to development initiation cycle time
  • Retention/Employee turn over
  • Time to hire
  • Prospect identification cost
  • Prospect to hire conversion rate
  • Hiring cost
  • Training cost
  • Time to acclimation for new employees

And most of these really are measuring learning outcomes or Intermediate Factors in Learning (see also Intermediate Factors). 

Real business outcomes really fall into either: increased revenue or reduced cost.  The rest are accepted intermediate factors for the business such as improved time-to-market, customer satisfaction, employee satisfaction, etc.  And there can be lots of these intermediate factors.

So going back to a positioning question – I wonder if we shouldn't all be talking about specific kinds of business outcomes or intermediate factors that can be achieved?

Friday, May 1, 2009

Call for eLearning Demonstrations

One of the hardest things to find are real-life examples of different kinds of eLearning Solutions. On May 21, Learn Trends will be hosting a free online event where the people who have developed interesting eLearning solutions will demonstrate and briefly discuss what they've done.

This is an online call for people who can "demonstrate" their system. Even if you can't participate - please help me get the word out on this.

Because of my concerns about bandwidth - demonstrations will really be screen shot slides. Sorry. This makes it a bit harder to demonstrate 3D worlds, video based training, etc. But it can still be done - just screen shot it and plan to explain it.

Generally we are looking for examples of a variety of different kinds of solutions. We hope they will vary from practical solutions to common problems to leading edge solutions. Some example areas and this is by no means inclusive:
  • Self-Paced Courses
  • Performance Support Tools
  • Mobile Learning
  • Collaborative Learning
  • Social / Network Learning
  • 3D Learning
  • Games
  • Toolkits
  • Interesting Tools
We especially would like participants who are developing solutions inside of organizations to participate. These are most often the hardest examples to see.

To participate, you must be available May 21 from 9AM to Noon Pacific Time.

If you are interested, please send me an email (akarrer@techempower.com) with:
  • A brief description
  • A couple of example screen shots
We expect to have more examples than we can show and discuss in the 3 hours, so we will likely only be able to choose some of what is sent in. We will try to choose a variety of different examples that represent effective patterns.

We are also trying to come up with ways to effectively share examples that are not shown real-time during the session.

Two last things:
  1. Please help me spread the word about this.
  2. If you know of places that examples already are captured, please point me to those. Another aspect of this is beginning to capture examples that will be available online.