Previous Thread
Next Thread
Print Thread
#113082 - 04/10/18 10:00 AM Behavior of osd_file & sockets  
Joined: Mar 2015
Posts: 49
fulivi Offline
Member
fulivi  Offline
Member

Joined: Mar 2015
Posts: 49
Hi,

I'm working a bit on my ieee-488 remotizer. I recently came across a little problem with the behavior of osd_file. By default this class tries to fill an internal buffer as much as possible when reading. This buffer is then flushed whenever there's a write operation (to keep this form of cache consistent with the content of the file on disk).
This behavior is really problematic when a bitbanger is used for socket I/O (like I do in the remotizer). Whenever there's some outgoing data any input data is lost if it's waiting in the osd_file buffer.
I added a virtual function "is_cacheable" to osd_file that returns "true" for all kind of files but sockets.
When this function returns false the reading operations don't cache anything in buffer and write operations don't flush it (it's always empty anyway).
It look an harmless change to me but I'm asking for your opinion on the matter.
An alternative approach could be to explicitly set a non-cacheable flag whenever a bitbanger (or similar) is used to implement a protocol over a socket. This flag should reflect the fact that the two directions of a socket are not related at all (i.e. writing the output doesn't affect the input).

Thanks.
--F.Ulivi

PS: I wrote an external "Amigo" drive emulator that can boot the HP64000 off an emulated 9134B hard-disk! More details to follow..

#113087 - 04/10/18 02:33 PM Re: Behavior of osd_file & sockets [Re: fulivi]  
Joined: Feb 2004
Posts: 1,939
Vas Crabb Offline
Very Senior Member
Vas Crabb  Offline
Very Senior Member

Joined: Feb 2004
Posts: 1,939
Sydney, Australia
Hang on, osd_file isn't a class, it's an interface that exposes a couple of static member functions that return implementation depending on the OS you built for and the path you pass in. The virtual methods in the interface are read, write, truncate, and flush. The socket implementation for POSIX is in src/osd/modules/file/posixsocket.cpp and you'll see that posix_osd_socket inherits osd_file directly. I can't see any cache in this class.

So there can't be any truth to the statement, "I recently came across a little problem with the behavior of osd_file. By default this class tries to fill an internal buffer as much as possible when reading." The osd_file class doesn't have any non-static member functions that aren't pure besides the destructor, and posix_osd_socket doesn't cache anything.

Can you please explain what you mean and link to the code?

#113089 - 04/10/18 03:00 PM Re: Behavior of osd_file & sockets [Re: fulivi]  
Joined: Mar 2015
Posts: 49
fulivi Offline
Member
fulivi  Offline
Member

Joined: Mar 2015
Posts: 49
Ok, sorry, the info I provided wasn't very accurate.
The caching happens in the core_osd_file objects (m_buffer is the cache). I added this function to osd_file:
Code
virtual bool is_cacheable() const { return true; }

This function is overridden in posix_osd_socket to return false.
Basing on the return value core_osd_file::read fills or not m_buffer. I changed the code in this way:
Code
	// if we're within the buffer, consume that first
	if (m_file->is_cacheable() && is_buffered())
		bytes_read += safe_buffer_copy(m_buffer, offset() - m_bufferbase, m_bufferbytes, buffer, bytes_read, length);

	// if we've got a small amount left, read it into the buffer first
	if (bytes_read < length)
	{
		if (m_file->is_cacheable() && (length - bytes_read) < (sizeof(m_buffer) / 2))
		{


I agree that placing a non-pure function in osd_file isn't the best thing to do. The is_cacheable function should be defined pure there and overridden in all sub-classes of osd_file (for example in posix_osd_socket).

My question is: do you agree in general with this approach or should I do it in some other way, please?
Thanks.
--F.Ulivi

#113091 - 04/10/18 04:00 PM Re: Behavior of osd_file & sockets [Re: fulivi]  
Joined: Feb 2004
Posts: 1,939
Vas Crabb Offline
Very Senior Member
Vas Crabb  Offline
Very Senior Member

Joined: Feb 2004
Posts: 1,939
Sydney, Australia
(As an aside, there are big issues with the file functions besides this stuff. The way core_file thinks it's in text and binary mode at the same time is a big problem in itself, and effectively prevents things like making a nice std::iostream wrapper for core_file. The fact that all read and write operations on osd_file attempt to seek is also problematic. There are flawed assumptions all through these classes. The bitbanger is also one of the most poorly named things in MAME - it's a byte source/sink, nothing to do with bits at all.)

I don't think this is a great way of doing it. It's putting another piece of magic in the wrong place. Cache policy should be set at a higher level. If you use fdopen to create a FILE* wrapping an fd that represents a socket, you still get the cache policy of the FILE* from stdio. Really, the bitbanger needs a way to specify that it wants a non-blocking read, and it needs some way to distinguish between "would block" and "end of stream".

#113095 - 04/11/18 10:00 AM Re: Behavior of osd_file & sockets [Re: fulivi]  
Joined: Mar 2015
Posts: 49
fulivi Offline
Member
fulivi  Offline
Member

Joined: Mar 2015
Posts: 49
If I understand this correctly, you are saying that to properly add this "no cache" thing to the file infrastructure, some major rethinking and refactoring are needed first. Honestly, I don't think I'm very qualified for such task.
Let's say that without "is_cacheable" the ieee-488 remotizer works most of the times as input and output operations seldom overlap. So, for the time being I'm not going to touch osd_file & co. When the remotizer works, it works and when it doesn't, bad luck...
--F.Ulivi

#113097 - 04/11/18 01:55 PM Re: Behavior of osd_file & sockets [Re: fulivi]  
Joined: May 2007
Posts: 534
mizapf Offline
Senior Member
mizapf  Offline
Senior Member

Joined: May 2007
Posts: 534
Germany
I remember a similar (possibly the same) problem when I implemented the serial interface on the TI-99 some years ago. We used a socket connection to send and receive serial data; I wrote an external program that served as a bridge between a socket and the real UART. The problem I had was that the emulated TMS9902 UART has no buffers, and was immediate run over on incoming data, although they were sent at the correct rate. The receiving socket, though, did not return single bytes but chunks of bytes. What I did was to get the bytes from the socket, queue them in a buffer, and poll that buffer at the expected rate (i.e. I calculated a time period between polls that corresponds to the baud rate). Works nicely, but hardware handshaking cannot work this way because you cannot ensure the synchrony between the data and handshake lines.


Who's Online Now
0 registered members (), 18 guests, and 3 spiders.
Key: Admin, Global Mod, Mod
Shout Box
Forum Statistics
Forums9
Topics8,525
Posts111,253
Members4,792
Most Online225
May 26th, 2014
Powered by UBB.threads™ PHP Forum Software 7.6.0
Page Time: 0.033s Queries: 14 (0.012s) Memory: 4.9694 MB (Peak: 5.1614 MB) Zlib enabled. Server Time: 2018-04-21 04:01:49 UTC