Right now it complains about their not being a local gatsby install and throws out a bunch of cryptic errors and then quits. I was helping a friend setup Gatsby and they got stuck for a bit as they hadn't entered the new gatsby directory after running gatsby new.
It's now been ~6 months since we added support for Gatsby being a local install. I think at the next minor version we should enforce this and not run gatsby develop/build unless gatsby can find itself (haha) in node_modules.