Test all targets

[toch]

@hone @zzak After different tries and research, please find here a solution to test the final bins on different targets (nix, osx, and win) for 64bits arch.

It works as following:

• if the test passes, the bins are pushed on a branch of the same name on the repo toch/mruby-cli-bins
• that new push triggers a build on Travis CI and Appveyor

as further steps, I'd like to:

• test 32 bits arch
• run bintests there too
• push on the github status of the mruby-cli

I think we can even generalize that to make it available to mruby-cli apps

PS: if ok for you I can squash the whole ofc

[zzak]

This is a really interesting way to test the binaries.

My main concern is the feedback loop is kind of broken, because we have to monitor a separate repo to see pass/fail -- and then manually(?) make the connection for which branch/pr triggered it.

I'm not sure there is a way to update the build status of another project within the bounds of github/travis.

Still, this is a really interesting attempt and I'm sure there are parts of this we could find really useful!

Thank you @toch!!

[toch]

My main concern is the feedback loop is kind of broken, because we have to monitor a separate repo to see pass/fail

I agree. I'd love to test binaries directly from the original repo. Unfortunately, I didn't find a proper way right now to do it, mainly because of the limitations of CI services:

• AppVeyor cannot (yet) run a docker container
• TravisCI cannot run a VM for OSX, and by doing so a docker container

and then manually(?) make the connection for which branch/pr triggered it.

for now, I only support internal branches. For PRs, I'd need to give access to that mruby-cli-bins repo to anybody, which I'm not really comfortable for now. That would need more careful considerations.

Anyway, for internal branches, I push on a branch with the same name. If we supported PRs, my idea was to push on a branch named pullrequest-<number>.

I'm not sure there is a way to update the build status of another project within the bounds of github/travis.

We could use the Github API to add a new status. This is a nice idea. That would close the loops in fact.

Still, this is a really interesting attempt and I'm sure there are parts of this we could find really useful!

If I can add a Github Status to the repo mruby-cli from tests run on mruby-cli-bins, would it be enough to be accepted? or do you see something else that should be changed?

[toch]

@hone The results from mruby-cli-bins are now pushed here too :)

it's good to be merged I think.