Re: Promoting Lucid Testing using KVM

[Matt Zimmerman-3]

(moving to ubuntu-devel@)

On Thu, Nov 05, 2009 at 02:42:46PM -0600, Dustin Kirkland wrote:
> Along the lines of KVM testing, I have another suggestion of how KVM
> might be used to test the Lucid Desktop and Server...
>
> Update-Manager eventually pops up a message saying "There is a New
> Distribution Available..." etc. and offers to perform the upgrade of
> your entire OS, after we GA.  And then the masses update.  And people
> gripe on the forums about color schemes, and apps that disappeared or
> moved.  And then this gets picked up by "journalists" looking to drive
> website traffic by trashing the latest Ubuntu release...

I believe there is a way to sandbox the upgrade now, so it can be "test
driven" and rolled back if the user is not happy.

> I wonder if it would be useful to offer to install a desktop
> notification (through an opt-in package installation or toggle in
> Update-Manager) that tells Ubuntu users (who have VT capability on their
> CPU) know that there is a new Development Milestone ISO available, and
> offer to download and launch that new milestone ISO in a Virtual
> Machine.

Rick, Marjo and I are actually hoping to take this a step further in Lucid
by making it possible to do this in the cloud, instead of locally in KVM.
We've already done this for Server Edition, where you can start an EC2
instance using any daily build without even downloading anything.

We want to make the same experience possible with Desktop Edition.  No
downloading ISOs, no installation, just the latest development code running
on demand.

> We could "safely" expose non-developer Ubuntu users (still running 9.10)
> to our Lucid Alpha, Beta, and RC ISOs within a Virtual Machine. This
> won't help as much with hardware issues (video, sound, wireless, etc.).
> But could help tremendously with both Desktop and Server application
> space testing, design, user experience, artwork, documentation, etc.  We
> could really broaden our Lucid test base and get more feedback much
> sooner, rather than 2-weeks after GA.

We get a tremendous amount of feedback and testing of Desktop Edition during
development, but we could do with more early user testing of server
milestones.  I think a cloud solution is even more compelling for servers,
since they're command-line operated.  Why download hundreds of megabytes of
OS, and spend time installing it, when you just to see it running for a
little while?

> Using a new feature of KVM, you can actually boot ISOs over HTTP and
> FTP.  So users could actually do this without even downloading the whole
> ISO ahead of time (assuming reasonable download speed).
>
> If you grab the qemu-kvm currently in karmic-proposed, you can do:
> kvm -m 512 -cdrom
> http://mirrors.kernel.org/ubuntu-releases/8.04.3/ubuntu-8.04.3-desktop-amd64.iso
>
> And stream an Ubuntu ISO, over the Internet, without using a backing
> disk image -- running entirely in RAM.  It would be cool to see Ubuntu
> 9.10 users doing this en masse with the Karmic ISOs as they become
> available.

That's interesting.  I wonder what the performance is like for a typical
broadband user.

--
 - mdz

--
ubuntu-devel mailing list
ubuntu-devel@...
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

[Rick Spencer-2]

On Sun, 2009-11-08 at 13:27 -0800, Matt Zimmerman wrote:
>
> Rick, Marjo and I are actually hoping to take this a step further in Lucid
> by making it possible to do this in the cloud, instead of locally in KVM.
> We've already done this for Server Edition, where you can start an EC2
> instance using any daily build without even downloading anything.
>
> We want to make the same experience possible with Desktop Edition.  No
> downloading ISOs, no installation, just the latest development code running
> on demand.
In fact Dustin and I discussed on IRC on Friday, as well. We discussed
approaching the project in steps ...

Step 1: Wrap testdrive up with a gtk front end to get the project
started quickly with few dependencies. We've already added a gtk front
end to the proejct, and I'll try to get it working nicely tomorrow.

Step 2: Dailies available in our private cloud for Canonical folks to
poke at it, especially designers, etc... If we can't pull of dailies in
the cloud in an automated manner, we should at least manually host
images of major milestones.

Step 3: Dailies available on aws to the public. Perhaps user would have
to have aws accounts and pay, or perhaps not.

Steps 2 and 3 bring up the notion of a "testers" iso, that has test
cases and/or tools built in.

Cheers, Rick




--
ubuntu-devel mailing list
ubuntu-devel@...
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

[Dustin Kirkland-4]

On Sun, Nov 8, 2009 at 3:27 PM, Matt Zimmerman <mdz@...> wrote:
> Rick, Marjo and I are actually hoping to take this a step further in Lucid
> by making it possible to do this in the cloud, instead of locally in KVM.
> We've already done this for Server Edition, where you can start an EC2
> instance using any daily build without even downloading anything.
>
> We want to make the same experience possible with Desktop Edition.  No
> downloading ISOs, no installation, just the latest development code running
> on demand.

I agree that testing in the cloud would be interesting.  But does this
obviate enabling users to test on their local hypervisor too?

I mean, if some user is running a Karmic desktop/laptop with KVM,
perhaps they would rather rsync the daily ISO locally, and testdrive
it in a VM.

I see three drawbacks with running a desktop image in the cloud:
 1) cost -- users would need to pay real money to do this
 2) performance -- VNC over the WAN can be painful
 3) experience -- to run in EC2, we're going to need to bundle an
image, which will be similar, but not identical to the LiveCD ISO
experience

Certainly from a developer's perspective, I need multiple fast virtual
machines running locally at any given time; the cost of running every
VM I need in the cloud would quickly eclipse my EC2 budget, and the
performance impact of VNC over the internet would make this simply
impractical.

> We get a tremendous amount of feedback and testing of Desktop Edition during
> development, but we could do with more early user testing of server
> milestones.

Tremendous feedback on the Desktop Edition -- sure, agreed.  But
enough?  There's a vocal minority of users that trash every Ubuntu
Desktop release in the days just before and just after every release.
I'd like to remove some of the barriers that prevent them from
providing the constructive feedback they have within the windows when
can do something about it.

>  I think a cloud solution is even more compelling for servers,
> since they're command-line operated.  Why download hundreds of megabytes of
> OS, and spend time installing it, when you just to see it running for a
> little while?

A cloud-based solution is definitely more compelling for server.
Remove the arguments I have above regarding VNC, as SSH access is
certainly sufficient.  And I agree that we need more server testing.
Testdrive, though is geared more toward desktop testing which, I
believe, isn't yet well suited to testing in the cloud.

As for downloading hundreds of megabytes, I implemented testdrive to
use a cache, and rsync/zsync where possible, so these downloads should
be incremental in many cases.  If your ISO is up-to-date, you can
launch over and over and over again at zero network cost.

> That's interesting.  I wonder what the performance is like for a typical
> broadband user.

I keep a daily updated ISO mirror locally, so I can boot and run any
ISO over my gigabit LAN as if the ISO were local to the hard drive.
Really nice, in that I don't have to keep a dozen ISOs on my laptop.

I have tried this feature at coffee shops too.  It's functional,
though admittedly slower.  It is faster than downloading the whole
ISO, though, which is nice.  I can point KVM at an HTTP url and start
booting the ISO much quicker than waiting the 1.5 hours wget told me
it would take to download from said coffee shop.

:-Dustin

--
ubuntu-devel mailing list
ubuntu-devel@...
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

[Przemysław Kulczycki-2]

2009/11/9 Dustin Kirkland <kirkland@...>:

> On Sun, Nov 8, 2009 at 3:27 PM, Matt Zimmerman <mdz@...> wrote:
>> Rick, Marjo and I are actually hoping to take this a step further in Lucid
>> by making it possible to do this in the cloud, instead of locally in KVM.
>> We've already done this for Server Edition, where you can start an EC2
>> instance using any daily build without even downloading anything.
>>
>> We want to make the same experience possible with Desktop Edition.  No
>> downloading ISOs, no installation, just the latest development code running
>> on demand.
>
> I agree that testing in the cloud would be interesting.  But does this
> obviate enabling users to test on their local hypervisor too?
>
> I mean, if some user is running a Karmic desktop/laptop with KVM,
> perhaps they would rather rsync the daily ISO locally, and testdrive
> it in a VM.
>
> I see three drawbacks with running a desktop image in the cloud:
>  1) cost -- users would need to pay real money to do this
>  2) performance -- VNC over the WAN can be painful
>  3) experience -- to run in EC2, we're going to need to bundle an
> image, which will be similar, but not identical to the LiveCD ISO
> experience

How about running the Lucid ISO from an USB drive?
It would be easier to set up, it would be persistent and much faster
than LiveCD/DVD,
updates could be done just using apt, no need for any paid cloud
account, you only need to have a 4GB USB flash drive which don't cost
much these days.

--
## Przemysław Kulczycki >><< Azrael Nightwalker ##
# jabber: azrael[na]jabster.pl | tlen: azrael29a #
### www: http://reksio.ftj.agh.edu.pl/~azrael/ ###

--
ubuntu-devel mailing list
ubuntu-devel@...
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

[Rick Spencer-2]

On Mon, 2009-11-09 at 07:40 -0600, Dustin Kirkland wrote:

> I agree that testing in the cloud would be interesting.  But does this
> obviate enabling users to test on their local hypervisor too?
>
No, this is an *and* not an *or*. One key advantage of testing in the
cloud is the ability to get rapid feedback from non-technical stake
holders who are not able to run the current development version. While a
local VM would be helpful, being able to log into a remote instance and
get "instant" gratification would be strong.

As I say, getting to the daily in the cloud is additive, note a couple
more draw back of testing in the cloud:
>
> I see three drawbacks with running a desktop image in the cloud:
>  1) cost -- users would need to pay real money to do this
[Not if it's for internal users running from our private clous]
>  2) performance -- VNC over the WAN can be painful
>  3) experience -- to run in EC2, we're going to need to bundle an
> image, which will be similar, but not identical to the LiveCD ISO
> experience
4) No ability to test hardware compatibility
5) No compositing (I am assuming) so harder to test for important "look
and feel" issues.

Cheers, Rick


--
ubuntu-devel mailing list
ubuntu-devel@...
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

[Martin Pitt-4]

Dustin Kirkland [2009-11-09  7:40 -0600]:
> Tremendous feedback on the Desktop Edition -- sure, agreed.  But
> enough?

At least we get about 50 times more bug reports than we can fix. :)

> There's a vocal minority of users that trash every Ubuntu
> Desktop release in the days just before and just after every release.

Most stuff that I saw hard to do with hardware specific regressions,
sound problems, or boot problems, i. e. precisely the bits which you
can't even test in a local VM properly, even less so in a cloud.

I do think that it would be very interesting to have these live online
systems, but it doubt that it will silence the complaints that we get
right now.

It would be great for people who work on applications, artwork, or
other designs, though.

> I have tried this feature at coffee shops too.  It's functional,
> though admittedly slower.  It is faster than downloading the whole
> ISO, though, which is nice.  I can point KVM at an HTTP url and start
> booting the ISO much quicker than waiting the 1.5 hours wget told me
> it would take to download from said coffee shop.

It's still painfully slow, though. Running rsync in the background
doesn't hurt really, but having to watch your kvm instance crawl to
the desktop does, since it has to download all the stuff while you are
watching it.

Martin
--
Martin Pitt                        | http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)

--
ubuntu-devel mailing list
ubuntu-devel@...
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

[Martin Pitt-4]

Przemek Kulczycki [2009-11-09 13:58 +0000]:
> How about running the Lucid ISO from an USB drive?
> It would be easier to set up, it would be persistent and much faster
> than LiveCD/DVD,

That's what I do, too. I keep a current image on my USB key which I
carry around on my keyring, so I have a fast ubuntu for hw testing or
other quick things everywhere.

Martin
--
Martin Pitt                        | http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)

--
ubuntu-devel mailing list
ubuntu-devel@...
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel