Use the "PostQuitMessage(0)" function in the stand alone message handler. As far as I know there isnt a direct BB or VSL code to quit a stand alone application from the VMO file. So you will need to do this from the stand alone code instead.
You could change/set a attribute value when pressing a button in the vmo file, and let the stand alone application check for that attribute every render/update cycle, to make the application quit when you press the button. I dont use the stand alone application so I dont know how its done in practice, but it should be documented in the SDK documentation.