Setup News Ticker
Geeking with Greg
Jun 24, 2017

Two decades of recommendations
IEEE Internet Computing just celebrated its 20th anniversary.

On its 20th anniversary, the editorial board created its first ever 'The Test of Time' award. I'm honored to say they gave it to our 2003 article, ' Recommendations: Item-to-Item Collaborative Filtering', which continues to be accessed, cited, and used in industry and research many years after its original publication.

In addition, for the 20th anniversary issue of IEEE Internet Computing, we wrote a new article, 'Two Decades of Recommender Systems at'. Some excerpts: launched item-based collaborative filtering in 1998, enabling recommendations at a previously unseen scale for millions of customers and a catalog of millions of items. Since we wrote about the algorithm in IEEE Internet Computing in 2003, it has seen widespread use across the Web, including YouTube, Netflix, and many others.

The algorithm's success has been from its simplicity, scalability, and often surprising and useful recommendations, as well as desirable properties such as updating immediately based on new information about a customer and being able to explain why it recommended something in a way that's easily understandable.

What was described in our 2003 IEEE Internet Computing article has faced many challenges and seen much development over the years ... We describe some of the updates, improvements, and adaptations for item-based collaborative filtering, and offer our view on what the future holds for collaborative filtering, recommender systems, and personalization.


What does the future hold for recommendations? ... Discovery should be like talking with a friend who knows you, knows what you like, works with you at every step, and anticipates your needs.

Recommendations and personalization live in the sea of data we all create as we move through the world, including what we find, what we discover, and what we love ... Intelligent computer algorithms leveraging collective human intelligence ... Computers helping people help other people.

The field remains wide open. An experience for every customer ... offering surprise and delight ... is a vision none have fully realized. Much opportunity remains to add intelligence and personalization to every part of every system, creating experiences that seem like a friend that knows you, what you like, and what others like, and understands what options are out there for you.

Geeking with Greg
Apr 30, 2017

All Crunchzilla tutorials now open source
All the code is now available for all the Crunchzilla coding tutorials.

Code Monster, Code Maven, and Game Maven from Crunchzilla have been used by hundreds of thousands of people around the world to experiment with learning to write computer programs.

There have been many requests to make them and available in languages other than English.

By open sourcing the Crunchzilla tutorials, I hope three things might happen:

Translations: I hope others are able to take the content and translate part or all of it into languages other than English for use in more classrooms around the world.

New lessons: New tutorials might teach programming games, working through puzzles or math problems, or perhaps a more traditional computer science curriculum aligned with a particular lesson plan.

Entirely new tutorials: Some of the ideas and techniques -- including the step-by-step learn-by-doing style, live code, informative error messages, and avoiding infinite loops in students' code -- might be useful for others. The code was designed to be all static, so you can easily create your own version just by editing the files and then putting all the files together on your own server. There is a single JSON file that contains all the lesson content.

If you use the code for anything that helps children learn, I'd love to hear about it (please e-mail me at

Geeking with Greg
Apr 01, 2017

Book review: Radical Candor
This just came out, the book Radical Candor by Kim Scott. It's a good read on managing and focused on people. I'd recommend it if you are a manager or help others manage people.

I'd summarize it by saying it takes a teaching and mentoring approach to management, very much of the school that managers primarily exist to help the people on their team. The advice is both practical and actionable, with specific advice for running 1:1s and meetings, and focused how to encourage conversations where people strive to improve themselves as well as helping others.

Some carefully selected quotes from the book:

'It seems obvious that good bosses must care personally about the people who report directly to them ... And yet ... '

'It turns out that when people trust you and believe you care about them, they are much more likely to accept and act on your praise and criticism, tell you what they really think about what you are doing well and, more importantly, not doing so well, engage in this same behavior with one another ... embrace their role on the team, and focus on getting results'

'When you're the boss, it's awkward to ask your direct reports to tell you frankly what they think of your performance, even more awkward for them than it is for you. To help, I [ask] ... 'Is there anything I could do or stop doing that would make it easier to work with me?' ... It is essential that you ... commit to sticking with the conversation until you have a genuine response. One technique is to count to six before saying anything else, forcing them to endure the silence. The goal is not to bully but to insist on a candid discussion ... Then listen with the intent to understand ... Once you've asked your question and embraced the discomfort and understood the criticism, you have to follow up by showing that you welcome it. You have to reward the candor if you want to get more of it ... Make a chance as soon as possible ... show you're trying.'

'If you can absorb the blows, the members of your team are more likely to be good bosses to their employees when they have them ... The rewards of watching people you care about flourish and then help others flourish.'

'The ultimate goal of Radical Candor is to achieve results collaboratively that you could never achieve individually ... A culture of guidance ... An exemplary team ... self-correcting quality whereby most problems are solved before you are even aware of them ... Don't start by bossing people. They'll just hate you. Start by listening to them.'?

Geeking with Greg
Dec 18, 2016

Quick links
Some of the tech news that caught my attention lately: Humans working for the AI: How we get ground truth for machine learning ([1])

Deep learning helping on diagnostic medical imaging with accuracy at human level ([1] [2] [3] [4])

BHAG from Intel: 'Intel aims to deliver up to 100x reduction in the time to train a deep learning model over the next three years compared to GPU' ([1])

Deep learning's success is mostly a lot of data paired with an algorithm that can take advantage of a lot of data ([1])

Fun! 'A software platform for evaluating and training intelligent agents across the world''s supply of games, websites and other applications ... Agents use the same senses and controls as humans: seeing pixels and using a keyboard and mouse.' ([1])

Details on Duolingo's learning algorithms, including that they found what worked best for students using A/B tests ([1])

A rant on hype-driven development ([1] [2])

Building finished products is hard ([1])

'There is an optimal newness for ideas -- advanced yet acceptable' ([1])

Massive expansion of Facebook in Seattle. Seattle is increasingly becoming a mini Silicon Valley ([1] [2])

Andy Jassy optimistic Amazon Web Services will become a $100B business ([1])

Detailed comparison of pricing on Google, AWS, and Azure. To summarize, it's complicated, and what your cheapest option will be depends a lot on what you're doing. ([1])

'Google has been carbon neutral since 2007, and [in 2017] we'll be powered by 100% renewable energy as our newest wind and solar farms come online' ([1])

Likely to see truly massive wind turbines in the near future ([1])

'The Waffle House Index also stands for something less obvious. It is an indicator of how complex and long supply chains are — for food, for fuel, for power — and of what it takes to plan around infrastructure that can be fragile in unexpected ways.' ([1])

Xkcd: 'Of course, 'Number of times I've gotten to make a decision twice to know for sure how it would have turned out' is still at 0.' ([1])

'Not one, nor tw

Geeking with Greg
Dec 10, 2016

Book review: Chaos Monkeys
Cynical, mercenary, and dark, this book aptly serves as an opposing view for any idealism you may have been feeling about Silicon Valley startups or their bigger brethren. Some of us work in technology to make a difference. That is not what you will find in this book.

It is a tale of a startup that wasn't really a startup, three people with no real product acquired after 10 months. It is a tale of sales and personal marketing, spinning unfavorable realities into golden-sounding tales capable of jumping the next hurdle and moving on to the next deal. It is a tale of greed and personal ambition, everything viewed through a Wall Street lens of climbing a hierarchy of wealth and power, some in the world of venture capital, and particularly detailed at Facebook.

Facebook comes out of the book particularly poorly, as if Zuck is a some kind of fickle boy king holding court with his sycophants. During his time at Facebook, the author appears to try to join this clique, only to grow bitter when entry is rebuffed.

Most interesting is the description of Facebook's struggle with advertising revenue, especially after its IPO. As the author describes it, Facebook couldn't figure out how to make the promised revenue. Eventually, in mid-2013 or so, they found a way, not by using data on what people do, but knowing who most people are, which turned out to be particularly important on mobile ('basic targeting like age and gender was a godsend to data-starved marketers ... data-wise, you have a first-party relationship with [only] a few apps'). The real value of Facebook turned out not to be its data on what people are doing, but merely being able to identify most people consistently and willing to exploit that to its fullest.

It helps if you know at least a few of the personalities featured in the book. Paul Graham, Sam Altman, Chris Sacca, Greg Badros, and many others make at least brief appearances, usually to get splattered with the slime that drips from these pages. Many VCs and people at Facebook and Twitter are also mentioned, mostly described as the amoral who's who of the rich and powerful of Silicon Valley.

Like many who got lucky, the author confuses luck with skill. Sure, that pitch meeting went well, but that meeting almost didn't happen. Success often was a result of a chance connection at the right time. In cases where the author angered someone with his arrogance or foolishness, someone should have killed the deal, and might have had they been in a slightly different mood that day. This startup was almost stillborn, barely making it into Y-combinator. The acqui-hire almost didn't happen, almost killed by lack of customer growth and shenanigans by the author. That everything worked out even as well as it did was mostly good fortune.

To his credit, the author realizes some of this in the end. In the acknowledgments, he writes, 'Let's be blunt: ours was a relationship of pure convenience, and I exploited you as much as you did me.' But he also writes of some he encountered, 'In a Valley world awash with mammoth greed and opportunism masquerading as beneficent innovation, you were the only real loyalty and idealism I ever encountered.' I'd like to think mammoth greed and opportunism has much smaller representation than idealistic innovation.

Some may call me wishful, but I think pushing for that idealistic world to be true is part of making it true. This book is not going to stop me from thinking that tech companies should be a force for idealistic innovation and promise for th

Geeking with Greg
Aug 28, 2016

More quick links
A tightly curated list of what has caught my attention lately: New Yorker on AI: 'A lot of what people are calling 'artificial intelligence' is really data analytics -- in other words, business as usual. If the hype leaves you asking 'What is A.I., really?,' don''t worry, you're not alone .... Intelligent software helps us interact and deal with the ... [information] onslaught ... winnowing an increasing number of inputs and options in a way that humans can''t manage without a helping hand .... A set of technologies that try to imitate or augment human intelligence .... [But] we are a long way from creating virtual human beings ... In the meantime, we're going to have to deal with the hyperbole surrounding A.I.' ([1])

Tim O'Reilly: 'Humans are increasingly going to be interacting with devices that are able to listen to us and talk back .... [Alexa] demonstrates that conversational interfaces can work, if they are designed right .... Smaller domains where you can deliver satisfying results, and within those domains, spend a lot of time thinking through the 'fit and finish' so that interfaces are intuitive, interactions are complete, and that what most people try to do 'just works'.' ([1])

Netflix: 'We think the combined effect of personalization and recommendations save us more than $1B per year' ([1] [2] [3])

'The main reasons cited for using ad blockers include avoiding disruptive ads (69%), ads that slow down their browsing experience (58%) and security / malware risks (56%). Privacy wasn''t the top answer. So Facebook thinks if its can make its ads non-interruptive, fast, [useful,] and secure, people won''t mind.' ([1] [2])

According to the NYT, Uber lost $1.2B on $2.1B in revenue in H1 2016 ([1] [2])

'Amazon reaches new high of 268,900 employees — skyrocketing 47% in just one year' ([1])

Amazon's going hard for Netflix on their key vulnerability, strength of the catalog ([1])

Great example of how Bezos sees failure as just a step toward success, following up on their $170M loss from an expensive Amazon Fire Phone with another (and I think very promising) attempt using existing cheap phones ([1] [2])

Talks from ScaledML 2016, including Jeff Dean, Qi Lu, Ilya Sutskever, and more ([1] [2])

Great paper on the data pipelines at Facebook and some of their design tradeoffs ([1])

Good article on Facebook's approach to research, not separate from engineering, not part of engineering, but just open (

Geeking with Greg
May 14, 2016

Code Monster from Crunchzilla is now open source
Code Monster from Crunchzilla is now open source, free to use and modify.

Code Monster is a tutorial that has been used by hundreds of thousands of children around the world to learn a little about programming. It's a series of short lessons where each lesson involves reading and modifying a small amount of code. Changes to the code show up instantly, students learning by example and by doing.

The lessons content for Code Monster from Crunchzilla is in a JSON file that can be modified fairly easily to create your own content. By open sourcing Code Monster from Crunchzilla, I hope three things might happen: Translations. Taking the current content and translating into languages other than English for use in more classrooms around the world.

New lessons and new content. By adding new messages and example code to the JSON lessons file, new tutorials could be created for teaching programming games, working through puzzles or math problems, or perhaps a more traditional computer science curriculum aligned with a particular lesson plan.

Entirely new tutorials. Some ideas and techniques used by Code Monster, such as how Code Monster provides informative error messages, how it does live code, or how it avoids infinite loops in students' code, might be useful for others creating web-based coding environments. Code Monster from Crunchzilla has been used in computer labs and classrooms around the world. One of the most common requests is translations into languages other than English. Now that the code is open source, I hope that makes it easier for translated and modified versions to get in front of even more children.

If you use the code for anything that helps children learn computer programming, I'd love to hear about it (please post a comment here or e-mail me at

Geeking with Greg
Mar 05, 2016

Virtual reality hitting the mainstream: The next $100 bet
Virtual reality is hot again, with dedicated hardware headsets launching from multiple manufacturers intended for general use.

The world is substantially different than the last time this happened. In particular, there's more computing power available in our smartphones than the most powerful graphics workstations had back in the 1990s. Google Cardboard and others take advantage of that, using a smartphone and little else for a quick-and-dirty virtual reality experience.

But, for a product to appeal to a broad market -- to get beyond early adopters with disposable income seeking to show something cool to friends a couple times -- it needs to survive the harsh judgement of busy people. It isn't enough for virtual reality on expensive dedicated hardware to mostly work. The experience will have to wow repeatedly at a price people like.

So, Daniel and I have another bet: "Virtual reality hardware (not counting cardboard) will not sell more than 10M units/year worldwide before March 2019." I'm saying it won't. Daniel says it will. Loser donates $100 to the winner's choice of charity.

Daniel already posted his side of the bet. In brief, he thinks three years will be enough time for someone to get it right.

I think that mainstream adoption of dedicated hardware for virtual reality requires breakthroughs in usability and price that are too difficult to achieve in the three year time frame. The experience just isn't good enough yet for it to be anything other than a toy for early adopters. Current virtual reality hardware is bulky, expensive, not fully immersive, and not addictive or compelling beyond the initial wow. I expect even the next generation will just be a niche market (low million units per year) until we see major developments on price, form factor, and quality of the experience.

There are several wild cards here. For example, it is possible that much cheaper units can be made to work. It's possible that someone discovers very carefully chosen environments and software tricks fool the brain into fully accepting the virtual reality, especially for gaming, increasing the appeal and making it a must-have experience for a lot of people. As unsavory as it is, pornography is often a wild card with new technology, potentially driving adoption in ways that can determine winners and losers. A breakthrough in display (such as retinal displays) might allow virtual reality hardware that is much cheaper and lighter. Business use is another unknown where virtual reality could provide a large cost savings over physical presence. I do think there are many ways in which I could lose this bet.

Like Daniel, I'll add some constraints to make my side of the bet even harder. I'd be surprised if dedicated virtual reality hardware sells more than 10M total over all three years. I'd also be surprised if virtual reality using smartphones (like Google Cardboard) goes beyond a toy, so, is used regularly by tens of millions for gaming, education, or virtual tourism.

And, like Daniel, I expect virtual reality to be big eventually, am frustrated by our current computing limitations, and think we should work to have much better from our computing devices today.

Geeking with Greg
Feb 27, 2016

Tablets replacing PCs: Resolving the $100 bet
In 2012, Professor Daniel Lemire and I bet $100 over the question of whether tablets would replace PCs.

Specifically, the bet was, "In some quarter of 2015, the unit sales of tablets will be at least twice the unit sales of traditional PCs, in the USA." Loser donates $100 USD to the charity of the winner's choice.

It's 2016, and tablet sales went far higher than I ever expected, approaching PC sales, roughly 60M/year units for both tablets and PCs in the US. But tablet sales seem to have peaked, with Q4 2015 unit sales worldwide actually 14% lower than the previous year, which is worse than the 8% decline in PC sales.

There are other surprises. One of my concerns was that a very cheap tablet would dominate the market, and Amazon did come out with a $50 tablet that got relatively good reviews and nearly tripled Amazon's market share on tablets. There hasn't been enough time yet to see what happens with very cheap tablets, but a tablets this cheap are a different category than the tablets that were around in 2012.

Another concern at the time was hybrid tablets, so tablets with detachable keyboards that function a lot like laptops, and whether they'd blur the line between PC and tablet. Hybrid tablets have done very well -- a major category in tablets -- and look likely to continue to grow over time.

The last concern at the time was whether tablets could thrive despite the pressure from increasingly larger and more powerful mobile phones. That seems to have been the biggest issue. Phablets are getting as large as early tablets, and tablets that try to be much bigger than a smartphone proved too unwieldy and sold poorly. After all, who needs a tablet when you've got a mobile that's almost as large?

The broader question in the bet was whether people would stop using PCs. PC sales have been in decline, though the pace of that decline has slowed recently. What seems to be happening is that people are continuing to use multiple devices, which was a visible trend back in 2012.

A phone is great when you want to do something quickly on the run. A bigger screen is good when you need to do a lot of reading. A keyboard, mouse, and large screen become useful when you're producing instead of consuming. If you need to do all of these, there's no reason to only have a phone, only a tablet, or only a PC. Instead, people often have all three and more.

Even though I technically won this bet, I want to congratulate Daniel Lemire on this getting much closer than I ever expected. I also admire the bravery he had to take the bet, especially with such favorable terms, and appreciate what I learned from this. The terms were that the loser donate $100 to the charity of the winner's choice, and I'd like to match the donation. Daniel and I will both be donating $100 to the Wikimedia Foundation, which runs Wikipedia.

Geeking with Greg
Jan 02, 2016

SwipeLingo and Javascript Notebook
I've been working on a couple educational projects since Google, SwipeLingo and Javascript Notebook. SwipeLingo is a quick matching game for touchscreens. Javascript Notebook is a tool for writing coding tutorials, exercises, and examples.

I'm unable to fully finish them and get them exactly where I wanted them before starting at Microsoft. But I'm launching anyway in case they or the ideas in them are useful to others.

is a game-with-a-purpose, a quick matching game that is both fun and helps like flash cards with memorization. There are example games — particularly interesting is Chinese numbers, where you learn the characters pretty quickly after starting with wild guessing — and it's also easy to create your own. I was motivated to create SwipeLingo by loving Duolingo but wanting the vocabulary memorization in it to be more fun, and also wanting to try to build a non-native touch web app game that works equally well across desktop, laptop, tablet, and phone.

tries to make it easy to write and share coding tutorials, coursework, examples, exercises, and experiments. It was heavily motivated by Stanford's CS101 class and their content. Here are some examples: "Getting Started", "Introduction to Programming", "What You Can Do". It's a bit like a simple Javascript-only IPython Notebook in feel, but runs entirely in the browser, requiring no configuration or set up, just write and share. Others can modify the code, run it, and save and share their own copies.

Please let me know if take a look and have any comments or suggestions. And please tell others who might be interested about them too!

  • CEOExpress
  • 1 Boston Place | Suite 2600
    Boston MA 02108
  • 617 482 1200
    617 299 8649 (fax)
  • Contact

©1999-2017 CEOExpress Company LLC