Recommend Error Messages

Proposal Details

• Who Updates: ...
• Date Proposed: 2011-08-10
• Date Last Updated: 2011-08-10
• Priority:
• Complexity: Low
• Focus: Usability

Current Situation

Vendor tools are often not LRM compliant. Error messages can be cryptic or non-existent.

Requirement

Recommend a standard format for error messages capturing key LRM content to promote clarity and standards compliance

Implementation details

Non normative appendix reference supplying phrases from semantic descriptions of features

Code Examples

Use Cases

Arguments FOR

• promotes the IEEE VHDL standard
• for compliance by tool developers - what diagnostics are missing
• for end users and tool developers - points to chapter and verse
• error messages capture key phrase from LRM content

Arguments AGAINST

-- JimLewis - 2013-11-27

Based on David's analysis below (2013-10-09), it will take a considerable effort to do this. Is someone willing to do the LRM work? Is someone willing to do the LRM work for free or pay someone to do this? Will vendors support it if the LRM does? If it is a significant effort for the vendors, the tool will cost more. Are we willing to pay more for this? Will you make a buying decision based on error reporting? In the next several revisions, I want features that will give me more capability. This is not one of them. This is a feature that makes it easier for newer users who have never used a compiler before. I think if you have used a compiler (any language) in the past, you can sort through the compiler speak to find the rood cause of errors. At least the compilers are giving error messages and not just failing during semantic analysis (most of the time anyway)

-- TristanGingold - 2014-10-21

In addition to Jim's comments, I would add that often VHDL text is cryptic for users. Are there commonly used language standard that specify errors ? What about internationalization ?

-- ErnstChristen - 2015-01-21

In support of Jim's comments, I'd like to add that in my experience users have a high desire to see diagnostic messages handled uniformly in their tool environment, regardless of where they come: VHDL or elsewhere. Vendor solutions will likely be driven by such requirements.

General Comments

-- DavidKoontz - 2013-10-09

I think I may have originally proposed this one. I had actually revisited the idea in the last couple of weeks having dealt with a large number of error issues by proxy. There's the obvious issue of of chapter and verse changing from one version of the LRM to the next. There's also a side issue that some errors aren't defined as such where you expect them, rather incidentally being described where other text places semantic restrictions. Defining errors can also be convoluted depending on specific peculiarities of language involving heavy use of the glossary.

I'm involved with development or maintenance of two VHDL tools. Going through the -1993 standard recently I found 133 instances of the text clearly conveying an error, and 11 occurances of language labelled as illegal. As a guestimate I think you could find somewhere between 2600 and 2900 instances of syntax errors in normative EBNF (found outside an appendix, see Jim Lewis's recent challenge in comp.lang.vhdl asking where the in the standard the parenthesis for separating different logical operators in an expression is required, the EBNF is normative and effectively requires let recursion in intermediary representation), some number of those could be collapsed by category. Failures likely ought to classified as lexical, syntax, semantic, elaboration and run time. All told I'd guess you could find somewhere around 4500 definable errors in -1993 VHDL. You could imagine that number might be somewhat higher in -2008 or -201X. Maybe twice as high.

The original two validation suite efforts for VHDL appear to have suffered from biased tool usage producing test cases and peculiar selection of testable sentences from the -1993 standard. They can't be relied on as definitive as a starting point in defining identifiable errors, plus there would be the issue of translating chapter and versus forward. The larger of the two sets identified less that 2600 errors, and maybe 1750 reliably. These would exclude most lexical errors, which seem to plague VHDL users inordinately. The issue being that if you don't address the needs of users a 7 percent solution isn't reaching the largest audience (really, there may be less than 180 people actually involved directly in language related tool developement world wide today, the success of VHDL in the market place requires users numbers vastly dwarfing the number of developers).

About now we can see a certain divergence between the needs of tool developers and end users. A certain vendor has fairly extensive error message system that is standard version-less, but doesn't help find authoratative answers in the LRM. At some level we can trust tool developers to have practiced their catechism faithfully, although as the size of the standard always seems to increase over time at some point that could become a sisyphean effort if it isn't already.

The real effort is in the standardization effort, defining and annotating all these errors accurately exceeds any recent efforts on the VHDL standard, being larger say than Ada's annotated reference manual. The effort could also be colored by compliance levels of tools used by the members of that subcommittee. Producing an appendix with text body references allows porting forward (or backward generally) between versions of the standard by changing pointers to the text and is reminiscent otherwise of the vendors error message system. The question becomes how thourough that appendix will (or can) actually be.

Tool developers would benefit primarily from forward references in the LRM body to the error message appendix (meaning it might want to be normative). There's also issue with an annotation scheme for errors that are only defined in the EBNF, a general problem with describing syntax.

You can also imagine that invalidating any vendors sytems with a less than full effort could encourage divergence similar to the IEEE library effort or otherwise not meet the intended needs.

-- DavidKoontz - 2015-02-23

I don't believe recommended error messages could be generated without the effort equivalent to reorganzing the standard. There appears to have been zero new implementations derived solely from the -2008 standard. This isn't a content issue, rather a lack or organized presentation.

Against

-- JimLewis - 2013-11-27

-- ErnstChristen - 2015-01-21

-- DavidKoontz - 2015-02-23

Add your signature above to indicate you reject the proposal

Supporters

-- ChrisHiggs - 2011-08-10

-- SharadSinha - 2013-10-09

-- Brent Hayhoe - 2013-10-09

(deleted support David.Koontz)

-- BrianDrummond - 2013-11-27

-- MortenZilmer - 2015-01-21

Add your signature here to indicate your support for the proposal