Right now, you can only insert images and save audio files when using the dictation feature from the mobile app. Google Keep's code includes multiple references to an upcoming feature that lets you upload any file to Google Drive and add it to a note, preview the file using the Google Drive Viewer or open it using various apps. You'll also be able to add videos (YouTube URLs or videos uploaded to Google Drive) and play them inside Google Keep.
While this isn't the Drive integration I expected, it's still a cool feature. Hopefully, Keep will become a Google Drive app and you'll be able to open and edit your notes from Google Drive.
Update: The screenshots are from the Chrome app for Google Keep, but there are similar references in the desktop Google Keep web app.