- User Since
- Jan 20 2014, 6:20 PM (192 w, 12 h)
Aug 7 2017
@jskladan we already did that; see above comments.
Aug 3 2017
Yeah, you do need to create separate PRs (at least we've never come up with a better workflow) - it can help to mention the other related PRs in each one, so we know they track together. If you can create the three PRs that'd be awesome, I'll try and find a minute to deploy them on staging and see how it goes. Thanks!
Aug 2 2017
The worker host is deployed now, I just haven't had the round tuits to test it out in any way. But the host is deployed and the workers show as registered, at least. Whoever set the host up added it to the *production* instance; we can easily switch it to the staging instance if we'd rather try things out there first.
Jun 27 2017
Jun 20 2017
Jun 13 2017
Jun 9 2017
Jun 1 2017
May 31 2017
May 19 2017
Looks fine. Out of interest - do you get a useful warning / error if you try to do an install *without* an ESP? If not, might be worth a bug report...
May 18 2017
Looks fine, assuming it really does create a btrfs volume with a / subvolume.
May 5 2017
May 4 2017
Thanks a lot for doing this! I've had it on my todo list approximately forever, but never got around to it.
I'd maybe call the first needle anaconda_blivet_size_unit-gib-selected , because we might find in future that some *other* units pre-selected, and we'll want another needle with the same tag called anaconda_blivet_size_unit-mib-selected or so.
Apr 28 2017
@jsedlak I have the exact same gut reaction to the flavors stuff, but I just couldn't think of anything obviously better :/ please do suggest something if you can, I'd love to see it look less weird. I'll have another look at it in a day or two with fresh eyes and see if I can come up with anything.
OK, now we've got the test cases and matrices adjusted and updating the reporting bits as well, so I think we can close this. I'll try and remember to keep an eye on things when the next validation event happens so we can check it all works as intended.
Apr 27 2017
This seems to be working - I deployed it to staging, and made a trivial edit to the description of https://bodhi.fedoraproject.org/updates/postgresql-9.5.6-1.fc24 in order to test it. The code worked exactly as intended:
Add a couple more packages to the whitelist (thanks sgallagh)
Apr 26 2017
Apr 22 2017
Ran fine on the 20170421.n.0 Fedora 26 nightly, so let's go ahead. Thanks again!
Thanks for the update. I've pushed the latest version to staging, we'll see how it runs.
Apr 20 2017
Oh, apart from some of the needles just not quite matching any more, one of them is a necessary *addition*: the anaconda_blivet_part_fs_xfs_selected needle. On Server images, since anaconda was fixed to set the right default filesystem for blivet-gui recently, the default filesystem will be xfs, while on other images, it'll be ext4. So we should have needles for both cases there. The tests will almost always run on Server images and so hit the xfs needle, but we should keep the ext4 needle just in case I guess.
OK, so testing this on Fedora 26 20170420.n.0 nightly I had to revise several of the needles, it seems like something changed a bit in current Fedora compared to whatever image you used to create the needles (fonts or GTK+ layout or something). You can see all the needles I had to add in the needles/ directory on staging, I'll leave them there for you to look at. Can you take a look at that and revise the needles in the diff? I don't think we'll need both versions, we only need the needles that work for the most recent 26 (and Rawhide, I guess). Thanks!
the anaconda_blivet_mountpoint_obscured needle actually indicates a bug, doesn't it - failure to include a scroll bar or something when the contents of that dialog are too tall for the space...have you reported that?
So far just some minor notes on read-through, overall looks fine. I'm going to deploy this to staging and re-run today's composes, to see how it behaves in the real world, before acking. Thanks a lot for the quick work on this, you rock!
Apr 19 2017
Heads up on this: we should be getting some PPC workers Real Soon Now, Dennis and Patrick are helping get a machine that IBM has kindly donated installed in PHX. Once it's available I'll deploy it as a worker host and we should be able to update this, merge it, and hopefully start running the tests.
Apr 18 2017
This is all done now, we got upstream to merge a ignore_failure flag and our relevant tests now use that, since e68e113f76dacd3bd14f94ab84687bc3784c255e .
"adding new symlink each time we want to run some module more than once?"
Apr 13 2017
Apr 5 2017
I've now written and landed the tests. For now, I'm not going to have them report to the wiki, as the test case includes some 'check the artwork and version numbers' requirements that openQA can't really do. I'm going to propose we separate the various artwork and version identification checks out from the test cases they're currently built in to, and make them into one or more separate test cases. That way openQA can report its results for the original test cases.
Mar 31 2017
Hmm, poked at this a bit today and it's really not trivial. The problem is getting a VM to behave anything like a real system under these conditions; they don't, really. nomodeset's effect on the qemu graphics 'devices' and drivers is rather different from how it acts on 'real' hardware and drivers.
Mar 30 2017
I've fixed the openQA bug in the 4.4-46 package build, just submitted to updates-testing. If you try again with that openQA package, it should work.
Mar 29 2017
It's an openQA bug:
yeah, actually I think I saw the same when testing with a manual tag. But I'm 99% sure the format of the tag is correct. It may be that the openQA we're running currently is broken, or just older than this feature? I'll take a look too.
Mar 28 2017
Mar 27 2017
Mar 25 2017
Fix and extend test
ack, looks fine to me, and I see the same bug.
Mar 24 2017
No idea. I think their idea is it was kind of an early implementation of 'soft failures' and they prefer the newer stuff that lets you explicitly trigger a soft failure or have a needle match trigger one, so they figured to get rid of a now-'useless' feature.
Mar 23 2017
Mar 21 2017
Mar 17 2017
Mar 16 2017
I know you can assign multiple tags to a single issue, but I prefer separate issues when the changes will be separate and could potentially land at different times...
Fix test issues noted by @mkrizek
Mar 14 2017
Mar 9 2017
Sounds great! We might need to do some documentation updates, in that case, though, to point to the new repo (I had changed our docs which mention Docker to point upstream on the assumption we were gonna upstream the changes). We may also need to send a PR for upstream docs to say 'if you want to run a containerized openQA for testing on Fedora, go here', or so.
Mar 8 2017
I dealt with openqa_fedora . Could we close this as a dupe of T863 for the remaining work to do on openqa_fedora_tools ?
Mar 6 2017
Don't use a pastebin as a permanent record, it expires :)
I still want it, it just got pushed down my priority list as we were out of the f25 blocker treadmill. Now we're on the f26 blocker treadmill I'll probably get back to this quite soon.
Mar 3 2017
I think we're OK to go with this for now, too, at least if I'm reading @kparal correctly - if anything, his remaining question is whether the test name should be in the scenario. But even if we decided to take it out in future, I don't think that's a problem, intended usages wouldn't be affected by the change I don't think. So I'm gonna go ahead and merge this.
The state of openqa_fedora_tools is correct: it still contains something that we can't get rid of yet, and the README correctly explains what. The docker bits in that repo are ahead of the ones upstream, but still a bit out of date. We're waiting for @jsedlak to update them and get the updated versions merged upstream, then we can ditch that repo entirely.
Feb 28 2017
I think I'm gonna call this resolved. We're now running the desktop and server post-install tests on every submission or edit of a critpath update.
Feb 27 2017
Rebase on return fix
sigh, of course. will fix that.
Feb 25 2017
Oh, forgot to mention - this diff will also make all ResultsDB reports include an arch extradata item (including the compose results, as well as the update results).
Make reporting work too
Feb 24 2017
Gah, it already wasn't supposed to, I messed up the commits. I'll adjust it.
Feb 23 2017
well, the tests we run cover *most* of the critpath functionality. I dunno how far I wanna go adding tests on this path, especially covering ground we could cover with Taskotron tests, but hey, why not. :) We *could* add a test that just does "dnf install *.rpm", I was thinking about that.
Feb 22 2017
And one more time, removing instead :)
Like the fedfind change, we should just kill this file.
In fact we now use Pagure for fedfind issue / PR tracking and I have retired the Phab project, so we should just remove this file entirely. I'll do that. Thanks. Please send future fedfind changes as Pagure PRs, not Phab diffs.
So I'm abandoning this as @jskladan doesn't like it. We'll take a different tack.
"Also, some of your testcases do not report arch (the upgrades as far as I checked) to resultsdb, but only have 64bit in testcase name - seems like a bug"
Feb 21 2017
I could live with that. For that approach, though, I don't know if I'd even bother creating the repo at this point; may as well just have the YAML or JSON or whatever file as part of the 'is it releaseable?' script for now. Since that will really be the *only* actual user at this point. If we decide we need some other user, but Policy Engine or whatever the real thing is called doesn't exist yet, we can split it out to a separate project at that point...
the real system is also going to have to deal with the problem of 'how do we synchronize the test names between the actual test systems and this system in a sensible way'? and now I think about it, it involves the 'test scenario' problem too! because test X on arch Y is not necessarily as important as test X on arch Z. Or test X on arch Y on image A vs. test X on arch Y on image B...