DpiProposal < P1076

P1076 Web>Vhdl2019CollectedRequirements>DpiProposal (2021-03-04, PeterFlake) (raw view)EditAttach
---+ Direct Programming Interface

<br /><br />%TOC%

---++ Proposal Details
   * Who Updates: Main.PeterFlake
   * Date Proposed:
   * Date Last Updated:2021/3/4
   * Priority:
   * Complexity:
   * Focus:

---+++ Introduction

The Direct Programming Interface is intended to facilitate the use of foreign language code in VHDL simulations. The VHDL Procedural Interface (VHPI) provides this capability in a very flexible way, but it requires a substantial amount of coding to perform simple operations. A similar situation exists in SystemVerilog, which has both a Direct Programming Interface and a Verilog Procedural Interface.

This proposal is in two parts. The first is to simplify calling foreign subprograms from VHDL. The second is to allow VHDL subprograms to be called from foreign subprograms.
---+++ Current Situation

A foreign model is either an entity whose architecture has the 'FOREIGN attribute (LRM clause 3.3) or a subprogram with the 'FOREIGN attribute (LRM clause 4.3). These must be distinguished from a foreign application, which is not named in VHDL, but interacts with the simulator via the VHPI. (LRM clause 20.1)

The 'FOREIGN attribute allows foreign language subprograms to be called from VHDL, but it does not formally allow foreign language data types returned by one subprogram call to be passed to the next subprogram call. This is because VHDL does not have a fully anonymous access type. Although an access type to an unbounded array may be used for this purpose, it is not following strict typing.

Clause 4.3 states that the 'FOREIGN attribute does not specify the parameter-passing mechanism, and that the format of the string is implementation dependent. However, clause 20.2.4 specifies these for VHPI subprograms, which have strings beginning with the reserved words VHPI or VHPIDIRECT. The VHPI parameter-passing mechanism requires extensive code to extract the actual values.

A foreign subprogram must have a body defined in VHDL even if it is never executed. (is this true?)

There is no mechanism for calling a VHDL subprogram from a foreign language.
---+++ Requirements

*For Part 1*

It should be simple to import a subprogram, i.e. have VHDL calling a C function without code to manipulate the parameters. This requires that data types of parameters can be implicitly converted between languages, rather than using access functions for explicit conversion. The imported subprogram should not be able to consume simulation time (*wait*).

The VHDL code should work without modification with any tool supporting the DPI standard. The C code should also work without modification. There is scope for discussion about whether the C code should work with any tool without re-compilation, or whether there can be a tool-specific header file.

*For Part 2*

It should be simple to export a subprogram, i.e. have a C function calling VHDL. This should take place in the context of an imported subprogram. The VHDL may be a procedure with a *wait*. If this is the case, the imported subprogram calling it must be a procedure, and may consume simulation time. Neither subprogram can have a *signal* parameter.

The DPI should support the Accellera SCE_MI interface to hardware emulators in a similar way to the SystemVerilog DPI. This requires not only the export of subprograms, but the ability to change context from one imported subprogram to another.

*Requests not addressed*

There has also been a request for VHDL to import a !SystemVerilog subprogram, and for VHDL to export a subprogram to SystemVerilog. However, the question of subprogram context adds complexity and so it is not included in this proposal.

There has also been a request to import Python, and dynamically determined subprograms.
---+++ Implementation details - Part 1

A new 'FOREIGN attribute string format is proposed. The string should include the language name, so that multiple languages can be supported by multiple attributes, as well as the linkage name and optionally the library name.

standard_direct_subprogram_binding ::=

VHPIDIRECT object_library_specifier execution_specifier

| DPI_C object_library_specifier execution_specifier

If the attribute is declared in a generic package, the string may be an expression containing a package generic, provided it is locally static (clause 9.4.2).

When a DPI_C imported subprogram is called, the instantiated scope ( i.e. which declaration in which instance or package called it) is recorded. This scope can be accessed and used for for calling VHPI functions. This functionality requires a new VHPI data type vhInstantiatedScope, as well as new VHPI functions vhGetInstantiatedScope and vhSetInstantiatedScope.

VHDL *input* mode parameters, which must be variables or constants, are passed to imported C functions by value (copy in) for scalars and by reference for aggregates. In the case of *out* or *inout* modes, which must be VHDL variables, the corresponding C formal parameter must also be a pointer. The actual in the call to C must be a pointer to a temporary variable provided by the tool. Note that the number and types of parameters are fixed, so that printf-style flexibility cannot be used (varargs). The value returned by an imported function is limited to a scalar type or a pointer.

The implicit mapping for scalar VHDL types to C99 types is:

BOOLEAN _Bool

BIT _Bool

CHARACTER unsigned char

INTEGER signed long long int

REAL double

TIME long long int

The implicit mapping from a VHDL STRING to a C unsigned char [ ] requires the tool to allocate memory and copy the string with a null character added to the end of a STRING for C format, before calling the C function. The tool should de-allocate the memory when the C function returns. Copying from C to VHDL, the actual parameter must either be a VHDL STRING, in which case the size is fixed, or an access type to a string, in which case the tool must allocate the memory.

In general, there can be a large performance penalty for using a similar approach for other types of arrays, so access macros and functions must be used for getting and setting values. These should be defined in a standard C header, like vhpi_user.h. A vendor may also specify the internal data structure to allow efficient implementation-specific access.

User-defined types such as enumerations need to be re-specified in C. It is up to the user to ensure that the formals in C match the formals in VHDL.

To pass a pointer between C functions in VHDL it is useful to have a special access type <b style="background-color: transparent;">chandle </b>that cannot be operated upon. This corresponds to void* in C.

Memory management must be done separately by each language. Memory allocated in C code must be freed in C code and memory allocated by VHDL must be garbage collected by the tool.

---+++ Code Examples - Part 1
<sticky> </sticky>

A function or procedure is imported as follows, with the string format standardized to pass the name to the linker:
<sticky><pre>
<B>package</B> P <B>is</B>

    <B>procedure</B> srand (seed:<B>in</B> NATURAL); -- C library void function
    <B>attribute</B> FOREIGN <B>of</B> srand: <B>procedure is</B> "DPI_C srand"; 

    <B>function</B> FFI_PUTS (s:STRING) <B>return</B> INTEGER; -- use Libffi to call C library
    <B>attribute</B> FOREIGN <B>of</B> FFI_PUTS: <B>function is</B> "DPI_C ffi_puts";

<B>end package</B>;
</pre></sticky>

The C code is as follows, with thanks to the Red Hat Libffi documention:
<sticky><pre>
#<B>include</B> &lt;stdio.h&gt;
#<B>include</B> &lt;ffi.h&gt;

<B>int</B> f( <B>int</B> x) { <B>return</B> x*x; }
<B>int</B> ffi_puts( <B>char</B> *s)

{
    ffi_cif cif;
    ffi_type *args[1];
    <B>void</B> *values[1];
    <B>int</B> rc = 0;

    /* Initialize the argument info vectors */

    args[0] = &ffi_type_pointer;
    values[0] = s;

    /* Initialize the call interface */

    <B>if</B> (ffi_prep_cif(&cif, FFI_DEFAULT_ABI, 1, &ffi_type_uint, args) == FFI_OK)

    ffi_call(&cif, puts, &rc, values); /* rc now holds the result of the call to puts */
    <B>return</B> rc;
}
</pre></sticky>
---++ Use Cases - Part 1

These enhancements will aid integration of a VHDL simulation with other tools and models, such as verification languages, instruction set simulators and hardware emulators. Libffi provides a means of integration with scripting languages.
---++ Implementation details - Part 2

A new attribute 'EXPORT is proposed for a subprogram, which has no effect on normal VHDL code but creates access for linking to a foreign language. In the case of C the attribute string should begin with DPI_C and contain the linkage name for the subprogram being exported. Note that the linkage name cannot be overloaded, so overloaded VHDL functions require a different linkage name for each set of parameter types. The attribute string can be a package generic, but it must be a locally static expression.

Exporting a VHDL function or procedure requires a unique name for the linker. If the export is declared in an instantiated package there is only one instance of the subprogram, so it can be called from anywhere. If the export is declared in an entity, or an uninstantiated package, there can be multiple instances of the subprogram. The tool has to select which one to use. The default is the instantiated scope (context) of the import declaration that called the C code, stored in vh ImportInstantiatedScope [ ]. This means that the exported subprogram is used by imported subprograms from the same entity instance. Note that an exported procedure can only be called by an imported procedure, and the exported procedure could call another imported procedure, so a stack of scopes is needed. To provide more flexibility the context (vh CurrentInstantiatedScope [ ], which defaults to vh ImportInstantiatedScope) can be changed by calling vhSetInstantiatedScope with a parameter of type vhInstantiatedScope obtained from a previous call to vhGetInstantiatedScope.

[Note that in SystemVerilog the names are svSetScope, svScope and svGetScope. A useful diagnostic tool is svGetCallerInfo (char **filename, int *lineNumber)]

An imported C procedure, that calls an exported VHDL procedure containing a *wait*, may be called by several VHDL processes at the same time, and therefore should be written to be multi-thread safe. Rules for re-entrant C coding include care with static variables and library calls.
---++ Code Examples - Part 2

A function or procedure is exported as follows, again with the attribute string to pass the name to the linker. It must be exported from a point in the package where the function name is visible.
<sticky><pre>
   <B>package body</B> Q <B>is</B>
      <B>function</B> G (x:INTEGER) <B>return</B> INTEGER <B>is</B>
      <B>begin</B>
         &hellip;
      <B>end</B> -- body
   <B>begin</B>
      <B>attribute</B> EXPORT <B>of</B> G: <B>function is</B> "DPI_C g";
   <B>end package body</B> Q;
</pre></sticky>

---++ 

<span style="background-color: #f6f6f6; color: #630000; font-size: 18.85px;">Use Cases - Part 2</span>

The Accellera SCE_MI interface specifies how a hardware emulator can be linked to a simulator.

<span style="background-color: #f6f6f6; color: #630000; font-size: 18.85px;">Arguments FOR</span>

These enhancements will make VHDL more useful as a simulation language because it will be easier and quicker to integrate it with other simulation and programming environments including operating systems and middleware.
---++ Arguments AGAINST

The functionality can be provided in the foreign language using customised code for wrapping subprograms and for interacting with the simulation using the VHPI.

Might be better to have a way to specify the linker level interfaces for the VHDL rather than defining stuff at user level, e.g. you just tell you VHDL compiler to import/export according to a particular <a href="https://en.wikipedia.org/wiki/Application_binary_interface" target="_blank">ABI</a> for given functions outside the language itself. Handling plumbing is usually easier on the C/C++ side.
---++ General Comments
   * Main.JohnShields - 2011-06-09 - Comments from Collect Requirements page
   * develop a DPI for VHDL to support easy creation of foreign procedures
      * aids integration with other tools
      * simple export/import of subp declaration
      * subprogram calling C function and vice versa
      * basic types can be marshalled
      * any ref type that need to be read/updated when assigned, when time is consumed require API (and we may choose not to support)
      * ?Interface between VHDL & Verilog/SystemVerilog as well as C?
   * Main.TristanGingold - 2015-08-07 - How do you make the difference between an imported subprogram and an exported subprogram ? Looks like for dpi_c, it's the presence of "extern" in the attribute. This is not very apparent, and I'd prefer to have two different attribute names (eg: C_IMPORT and C_EXPORT).
   * Main.TristanGingold - 2015-08-07 - I don't see the issue with the FOREIGN attribute (in order to import a subprogram. The language name can appear in the attribute as it is certainly strange to import one subprogram from two different implementation. The last sentence in 'current situation' ('the single attribute FOREIGN does not allow more than one foreign language') doesn't seem correct.
   * Main.TristanGingold - 2015-08-07 - It would be nice not to have to define the body (in vhdl) of an imported subprogram. Currently, even with FOREIGN, the user has to write a dummy body - which is stupid.
   * Main.TristanGingold - 2015-08-07 - Allowing suspension (wait statement) in exported subprogram may not be a good idea, given the implementation burden.
   * Main.TristanGingold - 2015-08-07 - The code example about file_open_kind is overly complex. I am not sure we want to map one type to a very different one.
   * Main.TristanGingold - 2015-08-07 - The first line of 'code example' is not valid VHDL!
   * Main.DavidKoontz - 2016-02-13 - int isn't a guaranteed 32 bit integer in C. Nor is VHDL integer guaranteed to be 32 bits (it's a minimum size). A long int isn't guaranteed to map to integer, either (and the same for unsigned long it). A VHDL string should map to an unsigned char[], and a character should map to an unsigned char. (the C char size is guaranteed to be at least 8 bits, VHDL uses ISO 8859-1, C 8 bit char is common).

I don't know that you have any business passing pointers across the C &lt;=&gt; VHDL interface. In !SystemVerilog for instance the C memory space and VHDL memory space are separate (that would infer you'd have to use PLI/VPI, or in our case VHPI). Protecting the simulator and model memory resources sounds like a _really_ good idea. C doesn't have what you could call a VHDL procedure equivalent with a parameter association list. Parameters of mode out imply an intermediary wrapper and passed pointers to a C function.

Note that VHDL can be otherwise implemented using interpretation rather than compiling and it's not outside the real of inserting foreign subprograms into a memory based model. That sort of implies worrying about implementation (e.g. IEEE Std 1800-2012 Annex J Inclusion of foreign language code) may be a bit out of bounds (which also makes it hard to claim a DPI-SV or the System C equivalent).

There's likely no easy (compact) way to pass VHDL context (scope and visibility based on where the FOREIGN declaration or specification is made) to a foreign language subprogram supplying the subprogram body (as much as Tristan hates the dummy bodies they could do that). I think Jim was commenting on something anecdotal from a particular vendor (starts with Mo) who wanted configurations to be analyzed into a different library than the entity and architecture they reference. This is the same issue, where does the context supplying scope and visibility come from? The way to recreate context is to re-analyze (even if you don't submit to a design library, the idea is that the design description is not updated nor is the reference library). In the case of DPI (or VHPI) how is that context conveyed? (like for a FOREIGN architecture, and ghdl doesn't do that yet.).

There are obvious issues with foreign subprograms containing wait statements which theoretically can cause an enclosing process to wait. That's real VHPI stuff instead, Signal parameters? What do signals look like? That's either implementation dependent or VHPI-ish and not Direct Programming Interface. Implementing signals likewise requires wrappers to do formal/actual translation between memory spaces for mode out (inout, buffer). Signals have different inward and outward references, read from an effective value, write to a particular driver's projected output waveform, for concurrent procedure calls there's an implicit sensitivity list. There's delay mechanism, pulse rejection limit, and after time expression plus some other out of band stuff (null, unaffected), and as Tristan notes do you really want call backs for waits in DPI?

There's bound to be a line you should draw between VHPI and DPI for feature support. Documenting could take more standard pages than !SystemVerilog uses, VHDL is more complex. There's also an issue with supported VHDL types. You need something extensible beyond supported types. How much the types for packages standard, std_logic_1164, numeric_bit and numeric_std should be predefined? This stuff needs some cerebral attention. You really ought to have an implementation before standardization. If it ain't working now it may be too late for -2017, depending on supported features.

-- Main.RyanHinton - 2020-02-17 -- Can we define distinct, meaningful levels of functionality? For example, I can imagine this facility without supporting *wait* statements in callbacks (e.g. only allowing function callbacks). Or a DPI without callbacks at all. Yes, this would be limiting relative to the current requirements, but is it sufficient? What are the actual use cases for the DPI?

-- Main.PeterLadow - 2020-02-29 -- The specific use can I can think of is to drastically simplify the interface for these simple types of operations. Eliminating the *wait* behavior and limiting parameters to simple types (i.e. no composites, access, etc) is sufficient I think for a first run at things. Consider your case of "puts". Now consider the VHPI boilerplate necessary to pass arguments and extract them:
<literal>&nbsp;</literal> <verbatim>PLI_VOID myputs(const struct vhpiCbDataS *cb_data_p)
{
  // Get the call handle
  vhpiHandleT call_hdl = cb_data_p->obj;
  // Parameter handles and iterator
  vhpiHandleT param_hdl, param_iter;

  // Return value from the function
  vhpiValueT ret;
  ret.format = vhpiInt;
  ret.value.intg = -1; // Default to an error

  // Verify there are parameters
  if ((param_iter = vhpi_iterator(vhpiParamDecls, call_hdl)) == 0)
  {
    // Do some error processing, perhaps vhpi_assert
  }
  else
  {
    // Iterate over the parameters
    while ((param_hdl = vhpi_scan(param_iter)))
    {
      // Verify the handle is of the correct type
      if (param_hdl.vValue->format != vhpiStrVal)
      {
        // Do some error processing, perhaps vhpi_assert
      }
      else
      {
        vhpiValueT val;
        // Zero out the value.  Note this will zero out val.bufsize, so that the call to vhpi_get_value will return the string length
        memset(&val, 0, sizeof(vhpiValueT));
        vhpi_get_value(param_hdl, &val);
        puts_retval = puts(val.value.str);
    }
    // Release the handle
    vhpi_release_handle( param_hdl );
  }
  // Not done yet!  Still need to set the return value.
  vhpi_put_value(call_hdl, &ret);
 }</verbatim>

That's just for a simple puts() call. Imagine something like "function check_bits(constant vec : in std_logic_vector; const mask : in std_logic_vector) return boolean;" Accesses to vectors, especially std_logic_vector (which is an array of an enumerated type) is brutal. Or consider this case "procedure log_data(const vec : in std_logic_vector; fd : integer; echo_stdout : boolean);" One needs a TON of boilerplate to parse parameters, extract values.

Now, if these could be simple C functions (as per Verilog's DPI):
<literal>&nbsp; int myputs(const char *str); int check_bits(const char vec[], int vec_len, const char mask, int mask_len); // Or something like this void log_data(const char vec[], int vec_len, int fd, int echo_stdout); // Or something like this &nbsp;</literal>

Significantly simpler for the user.
---++ Supporters

_Add your signature here to indicate your support for the proposal_

_-- Main.DavidSmith - 2012-09-26 David Smith Synopsys, Inc. - I support this as long as it is coordinated with 1076.1. We have had this request from user's of the language and I am pleased to see 1076 starting to address it._

-- [[Main.BrentHahoe][Brent Hayhoe]] - 2012-10-09

-- Main.RadoslawNawrot - 2015-04-28

<span style="font-size: 11.0pt; line-height: 115%; font-family: 'Calibri',sans-serif; mso-fareast-font-family: Calibri; mso-bidi-font-family: 'Times New Roman'; mso-ansi-language: EN-GB; mso-fareast-language: EN-US; mso-bidi-language: AR-SA;"><ins cite="mailto:peter%20flake" datetime="2020-12-28T16:02">The VH</ins><ins cite="mailto:peter%20flake" datetime="2020-12-28T16:03">DL procedural interface (VHPI) pro</ins><ins cite="mailto:peter%20flake" datetime="2020-12-28T16:04">vides this capability</ins><ins cite="mailto:peter%20flake" datetime="2020-12-28T16:07"> in a very flexible way</ins><ins cite="mailto:peter%20flake" datetime="2020-12-28T16:04">, but requires a </ins><ins cite="mailto:peter%20flake" datetime="2020-12-28T16:08">substantial</ins><ins cite="mailto:peter%20flake" datetime="2020-12-28T16:04"> amount of coding to </ins><ins cite="mailto:peter%20flake" datetime="2020-12-28T16:06">perform simple operations.</ins></span>

<ins cite="mailto:peter%20flake" datetime="2021-01-05T16:31">standard_direct_subprogram_binding ::=</ins>

<ins cite="mailto:peter%20flake" datetime="2021-01-05T16:32">VHPIDIRECT object_library_specifier execution_specifier</ins>

<ins cite="mailto:peter%20flake" datetime="2021-01-05T16:33">| VFFI object_library_specifier execution_specifier</ins>
Attachments
I	Attachment	Action	Size	Date	Who	Comment
pdf	Direct_Programming_Interface_for_VHDL_201x_Proposal_Update2.pdf	manage	93.6 K	2013-12-04 - 09:07	RadoslawNawrot	Direct Programming Interface for VHDL 201x Proposal Update uploaded. Co-existance with Sv and SystemC, type mapping, OpenArrays
Topic revision: r45 - 2021-03-04 - 17:46:35 - PeterFlake
P1076