the CSS Working Group needs you
By glazou on Saturday 25 September 2010, 16:35 - Standards - Permalink
(this message is posted with my CSS WG Co-chair hat on)
Yes, we need you. CSS 2.1 is a complex specification, and it has roughly 20,000 HTML4 and XHTML1 tests in its Test Suite. To make the document move from Candidate Recommendation to Proposed Recommendation, we need to show that each and every test in that Test Suite is passed by at least two different implementations. And that's where you can help :
if you have a few spare cycles and are able to test a few hundreds or thousands of the tests in the Test Suite with the latest version (see below) of Opera, Firefox4beta, IE or WebKit, please help us focusing on the least tested tests or the ones that have only 0 or 1 passing implementation.
The results are agregated into a database. Thanks a lot for your help!
Builds to be tested (and only those ones please):
Comments
We're still working on 2.1?? I thought we were moving on to 3!
Some of the tests are quite confusing. The descriptions of intended behavior don't match in any way what can be observed or even achieved with the markup AFAICT.
For example "there should be two identical boxes below" and all that can be observed is 8 "X" characters in groups of 2x2. A "box" in most other test cases mean a clearly visible rectangle of some sort.
One test had no description of intended behavior at all.
Fredrik: I generally agree, but when you see 'X's that probably means you haven't installed the "Ahem" font as the intro page tells you. In that font, an 'X' is a square.
But yeah, there is some questionable stuff.
BGW, right you are about not having the font installed. I should learn to read more carefully.
Currently going through the tests with the latest version of Firefox, but unfortunately encountering various failures, such as the run-in tests and some of the print media tests. Do you expect Firefox to pass all of these, or does it have some known failures and you just want all the passes registered?
It seems that many tests don't actually have any explanatory text saying what to check for; for example, "block-in-inline-remove-002" just has a blue-bordered box containing the text "One Two", but no explanation. I've skipped such tests.
Example of a failing print media test on Firefox: page-props-100-a. Properly shows four pages, but doesn't obey the "is entirely on the right half of the page" and "begins near the left edge of the page" requirements.
By the way, the print media tests would prove much easier if Firefox had a "This Frame -> Print Preview Frame" rather than just "This Frame -> Print Frame..."
I tried to review the results, and ended up at the page http://test.csswg.org/harness/resul... which says:
ERROR: Unable to obtain results.
File: /sites/csswg.org/test/htdocs/harness/lib_css2.1_harness/class.test_results.phi
Line: 131
Context: Array ( [test_suite] => CSS21_HTML [test_select] => [select_type] => 0 [browser] => [version] => [platform] => [grouping] => [modified] => [order] => 1 [sql] => SELECT testcase as Test, useragent as UserAgent, SUM(result='pass') as Pass, SUM(result='fail') as Fail, SUM(result='uncertain') as Uncertain FROM results GROUP BY testcase, useragentORDER BY count [r] => db_result_set Object ( [m_result_set] => ) [msg] => Unable to obtain results. )
I realize that it would not represent a test of CSS2.1 alone, but has anyone ever considered making a version of the CSS2.1 test suite which assumes support for @font-face, allowing it to pull in the Ahem font and other such test fonts automatically?