File Transfer and WinSock

Hi Mike,

Maybe you already know about this: there's a VFP sample at Microsoft.
http://support.microsoft.com/kb/315124

Richard Stecenko
Interactive Computer Services Inc.
Winnipeg, Canada
204.453.2052

MikeA

2009-12-30 18:50:47 UTC

I should have mentioned -that's why I'm looking for a sample that transfers
an entire file and not just a string of data.

Thanks,
Mike

Post by Richard Stecenko
Hi Mike,
Maybe you already know about this: there's a VFP sample at Microsoft.
http://support.microsoft.com/kb/315124
Richard Stecenko
Interactive Computer Services Inc.
Winnipeg, Canada
204.453.2052

Richard Stecenko

2009-12-30 19:16:15 UTC

Post by MikeA
I should have mentioned -that's why I'm looking for a sample that transfers
an entire file and not just a string of data.

Just for testing purposes, could you do a filetostr('MyFile') and then
send it using the Microsoft sample?

Richard Stecenko
Interactive Computer Services Inc.
Winnipeg, Canada
204.453.2052

Jürgen Wondzinski

2009-12-30 22:53:50 UTC

Hi Richard

Post by Richard Stecenko
could you do a filetostr('MyFile')

But that would limit him to 16Mb (the max length of a string in VFP), but he
wants to transfer files with 75Mb.
--
wOOdy
Visual FoxPro Evangelist
Microsoft "Most Valuable Professional" 1996 to 2009

"*´¨)
¸.·´¸.·*´¨) ¸.·*¨)
(¸.·´. (¸.·` *
..·`.Visual FoxPro: It's magic !
(¸.·``··*

MikeA

2009-12-31 01:42:13 UTC

Not just that but I could easily go beyond the 16mb by using the low level
VFP file i/o commands which I do in my application but socket code has to be
rewritten such that it checks for ready to send events, etc. I do all that
with winsock in my test program and it doesn't work. There are chunks of
data that are not being properly sent. A lot of times (but not always) the
chunks are being sent out as zeros in the buffer even though they are read
as non-zero bytes.

It's just perplexing.

Mike

Post by JÃ¼rgen Wondzinski
Hi Richard

Post by Richard Stecenko
could you do a filetostr('MyFile')

But that would limit him to 16Mb (the max length of a string in VFP), but
he wants to transfer files with 75Mb.
--
wOOdy
Visual FoxPro Evangelist
Microsoft "Most Valuable Professional" 1996 to 2009
"*Žš)
ž.·Žž.·*Žš) ž.·*š)
(ž.·Ž. (ž.·` *
..·`.Visual FoxPro: It's magic !
(ž.·``··*

Richard Stecenko

2009-12-31 02:14:31 UTC

Post by JÃ¼rgen Wondzinski
But that would limit him to 16Mb (the max length of a string in VFP), but
he wants to transfer files with 75Mb.

From the samples that I have seen, there is no send file method.
Instead, the client reads the file and sends it in packets. From the
samples that I have seen, the client sends a packet, the server echoes
back, and the client either re-sends or sends another packet.

In the Microsoft sample, all that is missing. It's primitive.

But if you look at the Microsoft sample, in the DataArival event (or
method), there's this code:

strData = SPACE(256) && Define string to pass to GetData
This.Object.GetData(@strData)
ThisForm.Edit1.Value = strData

Now, and I don't know how or why, if the getData() fails (screws up,
gets confused), then ThisForm.Edit1.Value is going to be a bunch of
spaces.

Mike says:
"A lot of times (but not always) the chunks are being sent out as
zeros in the buffer even though they are read as non-zero bytes."

So, it may be that the client reads the non-zero bytes and sends the
non-zero bytes, but the server fails (screws up, gets confused) and is
left with whatever strData was before the GetData.

So, it seems to me that it would be interesting to know what Mike has
in his code before the GetData in the string passed to GetData.

If for example, it was
strData = replicate('A', 256) && Define string to pass to GetData
This.Object.GetData(@strData)
ThisForm.Edit1.Value = strData
would he find As in the chunks?

If it's As then the problem is with the server, not with the client.
And probably some more sophisticated semaphoring is required.

Richard Stecenko
Interactive Computer Services Inc.
Winnipeg, Canada
204.453.2052

MikeA

2009-12-31 07:15:19 UTC

Here are my comments and I appreciate your continued advice and suggestions.

First, the Microsoft example has bugs in it for sending a file (or any
string of decent length). I have tested this even on a small file as little
as 7K. One must use the CreateBinary function when sending data or the data
will not match. This is not in the Microsoft example but is easy enough to
fix.

I have tried to modify the code so as to add some simple code to do a low
level file i/o and currently the file lengths are not matching in the
example with my modified code (when sending a 4 meg file). I'm still
working with this.

Post by Richard Stecenko
From the samples that I have seen, there is no send file method.

I understand that and that is why I'm looking for a simple program that
shows how to send a file. When I create my own and ttry, it is not working.

Post by Richard Stecenko
Instead, the client reads the file and sends it in packets. From the
samples that I have seen, the client sends a packet, the server echoes
back, and the client either re-sends or sends another packet.

These samples then are bad. There is no reason for this to be done at the
application layer because TCP/IP automatically verifies the packets and has
it's own checksums. I have written a program like that with pretty good
success (about 99% success) where I do echo a checksum and resend if
necessary. The problem with that is first it is already done at the TCP/IP
layer and second it's going to slow the transfer rate way way down (that's
the biggest problem) along with adding unnecessary overhead and code to the
program.

So my search continues but I appreciate your feedback and I'll check out the
Wiki samples.

Thank you again,
Mike

Post by JÃ¼rgen Wondzinski
But that would limit him to 16Mb (the max length of a string in VFP), but
he wants to transfer files with 75Mb.

From the samples that I have seen, there is no send file method.
Instead, the client reads the file and sends it in packets. From the
samples that I have seen, the client sends a packet, the server echoes
back, and the client either re-sends or sends another packet.
In the Microsoft sample, all that is missing. It's primitive.
But if you look at the Microsoft sample, in the DataArival event (or
strData = SPACE(256) && Define string to pass to GetData
ThisForm.Edit1.Value = strData
Now, and I don't know how or why, if the getData() fails (screws up,
gets confused), then ThisForm.Edit1.Value is going to be a bunch of
spaces.
"A lot of times (but not always) the chunks are being sent out as
zeros in the buffer even though they are read as non-zero bytes."
So, it may be that the client reads the non-zero bytes and sends the
non-zero bytes, but the server fails (screws up, gets confused) and is
left with whatever strData was before the GetData.
So, it seems to me that it would be interesting to know what Mike has
in his code before the GetData in the string passed to GetData.
If for example, it was
strData = replicate('A', 256) && Define string to pass to GetData
ThisForm.Edit1.Value = strData
would he find As in the chunks?
If it's As then the problem is with the server, not with the client.
And probably some more sophisticated semaphoring is required.
Richard Stecenko
Interactive Computer Services Inc.
Winnipeg, Canada
204.453.2052

Richard Stecenko

2009-12-31 16:51:52 UTC

I've been playing around with this for an hour.

I'm pretty sure that I'm sending the file in chunks.

But how do I get the server to stop receiving long enough to do
something with the chunks?

I keep getting an error of string too long. The server is just sitting
there on the getData until it chokes (at around 48 megabytes).

On the client side, do you send and then do a getData for a signal
from the server that it's ok to send the next chunk?

Richard Stecenko
Interactive Computer Services Inc.
Winnipeg, Canada
204.453.2052

MikeA

2010-01-01 10:25:42 UTC

You don't have get the server to stop receiving; the socket waits for the
server to accept the input in the buffer. Just save the data using either
strtofile or low level I/O. You can use either since strtofile will append
to a file but low level I/O is faster in the long run but for test purposes
it won't matter.

On the client side you just send and send - you don't wait for the server to
say it's okay to send the next chunk. You just send it and the client side
does have an error event that is fired if the socket on the sending side
says the buffer is full and it needs to wait. I played with this with
winsock and even with 75 megs when I have both client and server running on
my machine it never fires an error to wait but the mabry socket does fire an
error to wait due to buffer full. However, it's all irrelevant because when
the data arrives at the server and I write out the data it is corrupted
which is all part of my problem.

Thanks for the continued help and support as I hope to find a solution.

Mike

Post by Richard Stecenko
I've been playing around with this for an hour.
I'm pretty sure that I'm sending the file in chunks.
But how do I get the server to stop receiving long enough to do
something with the chunks?
I keep getting an error of string too long. The server is just sitting
there on the getData until it chokes (at around 48 megabytes).
On the client side, do you send and then do a getData for a signal
from the server that it's ok to send the next chunk?
Richard Stecenko
Interactive Computer Services Inc.
Winnipeg, Canada
204.453.2052

Richard Stecenko

2009-12-31 02:33:27 UTC

Here's a better sample:
http://fox.wikis.com/wc.dll?Wiki~WinSockOCXSample

Richard Stecenko
Interactive Computer Services Inc.
Winnipeg, Canada
204.453.2052

MikeA

2009-12-31 07:22:17 UTC

I looked a the Wiki link you sent - I've played around with EEVA and gotten
it to work nicely but that is a webserver app and it works great with VFP
for web apps. However, that has nothing to do with sending a large chunk of
data such as a zip file from a client app to a server app.

Still searching for a solution - all help is greatly appreciated.

Thanks,
Mike

Post by Richard Stecenko
http://fox.wikis.com/wc.dll?Wiki~WinSockOCXSample
Richard Stecenko
Interactive Computer Services Inc.
Winnipeg, Canada
204.453.2052

MikeA

2009-12-30 18:50:13 UTC

Actually I was not aware of that example. It's a great "starter." The
problem is that example only transfers a string of data. The problem I'm
having is when I transfer a large file like 75 megs or so and some of the
"chunks" of data are not being properly sent.

I'm stumped.

Thanks,
Mike

Gene Wirchenko

2009-12-31 05:15:58 UTC

On Tue, 29 Dec 2009 17:15:41 -0800, "MikeA"

Post by MikeA
Does anyone have sample code using winsock to transfer a file with a VFP
program clientsample.prg and serversample.prg? I'm still having problems
with no answers from prior posts and every time I think I'm that much closer
to a solution it just doesn't happen. I would think someone would have some
sample code somewhere. I searched the UT last night but found nothing and
I'm just getting a little tired of staying up til well after 3AM every
night.
All help would be greatly appreciated.

How is the data being sent, by TCP or UDP? The latter is not
guaranteed to be delivered.

Sincerely,

Gene Wirchenko

MikeA

2009-12-31 07:17:36 UTC

I'm supplying an IP address and port so would that mean TCP/IP? I have put
a sniffer on my network to check this out and I see WRONG DATA is being sent
out from the sender. I see packets of zeros are being sent every so often
so the problem is happening from the client side sometime before the socket
sends its data out it has the wrong data but I'm not sure why.

Thanks,
Mike

Post by Gene Wirchenko
On Tue, 29 Dec 2009 17:15:41 -0800, "MikeA"

How is the data being sent, by TCP or UDP? The latter is not
guaranteed to be delivered.
Sincerely,
Gene Wirchenko

Gene Wirchenko

2009-12-31 18:55:00 UTC

On Wed, 30 Dec 2009 23:17:36 -0800, "MikeA"

Post by MikeA
I'm supplying an IP address and port so would that mean TCP/IP? I have put

No.

IP Address, Port Number, and whether TCP or UDP are what are
needed. The calls that you use might establish a TCP or a UDP socket.
Which is it?

If the latter, you will have problems when the network gets
congested. You could have it at any time, but a non-malicious
implementation is not going to drop UDP data for no good reason. UDP
does not have guaranteed delivery, so it can be dropped, and there
will be no retry. TCP has retry.

Post by MikeA
a sniffer on my network to check this out and I see WRONG DATA is being sent
out from the sender. I see packets of zeros are being sent every so often
so the problem is happening from the client side sometime before the socket
sends its data out it has the wrong data but I'm not sure why.

How do you know that the zeroes are connected with your big file?

If you can be sure, then you have a problem up-line of
transmission.

[snip]

Bottom-posting makes it easier to follow technical discussions.
Please use it.

Sincerely,

Gene Wirchenko

MikeA

2010-01-01 10:32:32 UTC

I know for a fact the protocol with the Winsock is tcp/ip and I'm almost
positive it is TCP/IP with the mabry socket. However, it would not matter
because the problem is not guarantee of delivery. The problem is at the
sending side it is sending out every so often chunks of zeros and sometimes
other bad data - my sniffer detected this.

Mike

Post by Gene Wirchenko
On Tue, 29 Dec 2009 17:15:41 -0800, "MikeA"

How is the data being sent, by TCP or UDP? The latter is not
guaranteed to be delivered.
Sincerely,
Gene Wirchenko

MikeA

2010-01-03 08:11:31 UTC

Anyone have any ideas as to how to transfer a file through winsock? I don't
know where else to look or what else to do to troubleshoot this problem.

Thanks,
Mike

Post by MikeA
Does anyone have sample code using winsock to transfer a file with a VFP
program clientsample.prg and serversample.prg? I'm still having problems
with no answers from prior posts and every time I think I'm that much
closer to a solution it just doesn't happen. I would think someone would
have some sample code somewhere. I searched the UT last night but found
nothing and I'm just getting a little tired of staying up til well after
3AM every night.
All help would be greatly appreciated.
Thanks,
Mike

Man-wai Chang to The Door (24000bps)

2010-01-03 09:49:37 UTC

Post by MikeA
Anyone have any ideas as to how to transfer a file through winsock? I don't
know where else to look or what else to do to troubleshoot this problem.

I would use the built-in command-line ftp client....

--
@~@ Might, Courage, Vision, SINCERITY.
/ v \ Simplicity is Beauty! May the Force and Farce be with you!
/( _ )\ (x86_64 Ubuntu 9.10) Linux 2.6.32.2
^ ^ 17:49:01 up 13 days 1:03 2 users load average: 1.09 1.04 1.01
不借貸! 不詐騙! 不援交! 不打交! 不打劫! 不自殺! 請考慮綜援 (CSSA):
http://www.swd.gov.hk/tc/index/site_pubsvc/page_socsecu/sub_addressesa

Man-wai Chang to The Door (24000bps)

2010-01-03 13:59:46 UTC

Post by MikeA
Does anyone have sample code using winsock to transfer a file with a VFP
program clientsample.prg and serversample.prg? I'm still having problems

Try google "ftp using winsock" as well...

--
@~@ Might, Courage, Vision, SINCERITY.
/ v \ Simplicity is Beauty! May the Force and Farce be with you!
/( _ )\ (x86_64 Ubuntu 9.10) Linux 2.6.32.2
^ ^ 21:59:01 up 13 days 5:13 2 users load average: 1.14 1.05 1.01
不借貸! 不詐騙! 不援交! 不打交! 不打劫! 不自殺! 請考慮綜援 (CSSA):
http://www.swd.gov.hk/tc/index/site_pubsvc/page_socsecu/sub_addressesa

Jeff Grippe

2010-01-05 14:45:33 UTC

This is a more complex problem that fits a newsgroup posting but I can
explain some concepts and provide some samples.

I have message types defined and a packet protocol that I use which may be
too "heavy" for sending large files. I break up what I'm sending into
packets of a fixed length. I send one packet and wait for a confirmation
before sending the next packet. Each instance of my application has an IP
address and port which it listens on. All applications can both send and
receive.

You need to bind the winsock control to some port. I have multiple users
each of whom binds to a separate port. I begin with port 1002 and look for
an available port. Once a user has bound to a port, I save the results in a
table. I use UDP Protocol and I save all of my "messages" in a table. This
isn't necessary but I like having the trail. What I call a message is just a
string of characters. There's no reason why it couldn't be the contents of a
file.

*** PORT BINDING
this.nIPPort = 1002 && first port to try binding to
this.oMSWinSock.protocol = 1 && UDP
this.oMSWinSock.Bind(this.nIPPort)

*** ERROR HANDLING IN CASE THE PORT DOESN'T BIND
LPARAMETERS nError, cMethod, nLine
DO CASE
CASE nError = 1429 && Problem binding port
this.nIPPort = this.nIPPort + 1 && try next sequential port
RETRY
OTHERWISE
=MESSAGEBOX(STR(nError,4,0)+" "+cMethod+" "+STR(nLine,4,0)+MESSAGE())
ENDCASE

*** SENDING DATA
this.oMSWinSock.RemoteHost = lcToIP && whatever ip address you are
sending to
this.oMSWinSock.RemotePort = lnToPort && whatever port you are sending to

*** My particular message protocol
lcFullMessage = "<MESSAGE>"+;
"<"+ALLTRIM(STR(lnPacket,5,0))+">"+;
"<"+ALLTRIM(STR(lnPacketCount,5,0))+">"+;
"<"+this.oMSWinSock.localip+">"+;
"<"+ALLTRIM(STR(this.nIPPort,6,0))+">"+;
"<"+pcmessageid+">"+;
"<"+SYS(2007,lcMessageSegment)+">"+;
"|"+lcMessageSegment

*** Remember that I save my messages
SELECT SentMessages
APPEND BLANK
REPLACE type WITH "MESSAGE", ;
packet WITH lnPacket, ;
packetcount WITH lnPacketCount, ;
ToIP WITH this.oMSWinSock.RemoteHost, ;
ToPort WITH this.oMSWinSock.RemotePort, ;
MessageID WITH pcMessageID, ;
CheckSum WITH SYS(2007, lcMessageSegment), ;
MessageData WITH lcMessageSegment, ;
RawData WITH lcFullMessage

*** SENDING THE MESSAGE
this.oMSWinSock.SendData(lcFullMessage)

The message comes in to the WinSock control and fires the DataArrivalEvent.
This in turn uses the GetData method to return the data. That is where your
code has to go to pickup the data that was sent.

*** RECEIVING THE DATA IN THE DataArrivalEvent
*** ActiveX Control Event ***
LPARAMETERS bytestotal
lcData = SPACE(256) && Define string to pass to GetData
* get fields and data
This.GetData(@lcData)
lcType = STREXTRACT(lcData, "<", ">", 1)
*** Processing the data using my protocol
DO CASE
CASE lcType = "MESSAGE" && message
lnSplitPos1 = AT("|", lcData, 1)
lnPacketNumber = VAL(STREXTRACT(lcData, "<", ">", 2))
lnTotalPackets = VAL(STREXTRACT(lcData, "<", ">", 3))
lcIPFrom = STREXTRACT(lcData, "<", ">", 4)
lnPortFrom = VAL(STREXTRACT(lcData, "<", ">", 5))
lcMessageID = STREXTRACT(lcData, "<", ">", 6)
lcCheckSum = STREXTRACT(lcData, "<", ">", 7)
lcMessage = SUBSTR(lcData, lnSplitPos1+1, LEN(lcData) - lnSplitPos1+1)

*** only necessary if you want to save the data like I do. Very useful
for debugging
SELECT RecvMessages
APPEND BLANK
replace type WITH lcType, ;
packet WITH lnPacketNumber, ;
packetcount WITH lnTotalPackets, ;
FromIP WITH lcIPFrom, ;
FromPort WITH lnPortFrom, ;
MessageID WITH lcMessageID, ;
CheckSum WITH lcCheckSum, ;
MessageData WITH lcMessage, ;
RawData WITH lcData
IF lcCheckSum <> SYS(2007, lcMessage) && checksum mismatch
=MESSAGEBOX("Check Sum Error" + lcData)
* send Re-Transmit Request - Not yet coded. This hasn't happened yet
(fortunately)
ELSE
* send Packet Confirmation
this.Parent.SendConfirmation(lcMessageID, lnPacketNumber)
ENDIF

IF lnPacketNumber = lnTotalPackets && last packet received
*** if all packets received correctly
this.Parent.Messagereceived(lcMessageID)
ENDIF

CASE lcType = "CONFIRM"
lcMessageID = STREXTRACT(lcData, "<", ">", 2)
lnPacketNumber = VAL(STREXTRACT(lcData, "<", ">", 3))
SELECT SentMessages
SET ORDER TO MessageID
SEEK lcMessageID + STR(lnPacketNumber,5,0)
IF FOUND()
replace confirmed WITH .t.
ENDIF
SELECT RecvMessages
APPEND BLANK
replace type WITH lcType, ;
packet WITH lnPacketNumber, ;
MessageID WITH lcMessageID, ;
RawData WITH lcData
this.Parent.lastconfirm = lnPacketNumber
this.Parent.ConfirmationReceived(lcMessageID)

OTHERWISE
=MESSAGEBOX(lcData)
ENDCASE

Post by MikeA
Does anyone have sample code using winsock to transfer a file with a VFP
program clientsample.prg and serversample.prg? I'm still having problems
with no answers from prior posts and every time I think I'm that much
closer to a solution it just doesn't happen. I would think someone would
have some sample code somewhere. I searched the UT last night but found
nothing and I'm just getting a little tired of staying up til well after
3AM every night.
All help would be greatly appreciated.
Thanks,
Mike

MikeA

2010-01-05 16:12:38 UTC

Jeff - thank you for your post.

My code works fine when I don't send too much information. It also works if
I do send information and use my own ACK and NAK. But, I should not have to
do that. My only conclusion at this time is Winsock has bugs in the control
itself causing problems when sending a lot of data.

But, I'm still researching all this.

Thanks,
Mike

Post by Jeff Grippe
This is a more complex problem that fits a newsgroup posting but I can
explain some concepts and provide some samples.
I have message types defined and a packet protocol that I use which may be
too "heavy" for sending large files. I break up what I'm sending into
packets of a fixed length. I send one packet and wait for a confirmation
before sending the next packet. Each instance of my application has an IP
address and port which it listens on. All applications can both send and
receive.
You need to bind the winsock control to some port. I have multiple users
each of whom binds to a separate port. I begin with port 1002 and look for
an available port. Once a user has bound to a port, I save the results in
a table. I use UDP Protocol and I save all of my "messages" in a table.
This isn't necessary but I like having the trail. What I call a message is
just a string of characters. There's no reason why it couldn't be the
contents of a file.
*** PORT BINDING
this.nIPPort = 1002 && first port to try binding to
this.oMSWinSock.protocol = 1 && UDP
this.oMSWinSock.Bind(this.nIPPort)
*** ERROR HANDLING IN CASE THE PORT DOESN'T BIND
LPARAMETERS nError, cMethod, nLine
DO CASE
CASE nError = 1429 && Problem binding port
this.nIPPort = this.nIPPort + 1 && try next sequential port
RETRY
OTHERWISE
=MESSAGEBOX(STR(nError,4,0)+" "+cMethod+" "+STR(nLine,4,0)+MESSAGE())
ENDCASE
*** SENDING DATA
this.oMSWinSock.RemoteHost = lcToIP && whatever ip address you are
sending to
this.oMSWinSock.RemotePort = lnToPort && whatever port you are sending to
*** My particular message protocol
lcFullMessage = "<MESSAGE>"+;
"<"+ALLTRIM(STR(lnPacket,5,0))+">"+;
"<"+ALLTRIM(STR(lnPacketCount,5,0))+">"+;
"<"+this.oMSWinSock.localip+">"+;
"<"+ALLTRIM(STR(this.nIPPort,6,0))+">"+;
"<"+pcmessageid+">"+;
"<"+SYS(2007,lcMessageSegment)+">"+;
"|"+lcMessageSegment
*** Remember that I save my messages
SELECT SentMessages
APPEND BLANK
REPLACE type WITH "MESSAGE", ;
packet WITH lnPacket, ;
packetcount WITH lnPacketCount, ;
ToIP WITH this.oMSWinSock.RemoteHost, ;
ToPort WITH this.oMSWinSock.RemotePort, ;
MessageID WITH pcMessageID, ;
CheckSum WITH SYS(2007, lcMessageSegment), ;
MessageData WITH lcMessageSegment, ;
RawData WITH lcFullMessage
*** SENDING THE MESSAGE
this.oMSWinSock.SendData(lcFullMessage)
The message comes in to the WinSock control and fires the
DataArrivalEvent. This in turn uses the GetData method to return the data.
That is where your code has to go to pickup the data that was sent.
*** RECEIVING THE DATA IN THE DataArrivalEvent
*** ActiveX Control Event ***
LPARAMETERS bytestotal
lcData = SPACE(256) && Define string to pass to GetData
* get fields and data
lcType = STREXTRACT(lcData, "<", ">", 1)
*** Processing the data using my protocol
DO CASE
CASE lcType = "MESSAGE" && message
lnSplitPos1 = AT("|", lcData, 1)
lnPacketNumber = VAL(STREXTRACT(lcData, "<", ">", 2))
lnTotalPackets = VAL(STREXTRACT(lcData, "<", ">", 3))
lcIPFrom = STREXTRACT(lcData, "<", ">", 4)
lnPortFrom = VAL(STREXTRACT(lcData, "<", ">", 5))
lcMessageID = STREXTRACT(lcData, "<", ">", 6)
lcCheckSum = STREXTRACT(lcData, "<", ">", 7)
lcMessage = SUBSTR(lcData, lnSplitPos1+1, LEN(lcData) - lnSplitPos1+1)
*** only necessary if you want to save the data like I do. Very useful
for debugging
SELECT RecvMessages
APPEND BLANK
replace type WITH lcType, ;
packet WITH lnPacketNumber, ;
packetcount WITH lnTotalPackets, ;
FromIP WITH lcIPFrom, ;
FromPort WITH lnPortFrom, ;
MessageID WITH lcMessageID, ;
CheckSum WITH lcCheckSum, ;
MessageData WITH lcMessage, ;
RawData WITH lcData
IF lcCheckSum <> SYS(2007, lcMessage) && checksum mismatch
=MESSAGEBOX("Check Sum Error" + lcData)
* send Re-Transmit Request - Not yet coded. This hasn't happened yet
(fortunately)
ELSE
* send Packet Confirmation
this.Parent.SendConfirmation(lcMessageID, lnPacketNumber)
ENDIF
IF lnPacketNumber = lnTotalPackets && last packet received
*** if all packets received correctly
this.Parent.Messagereceived(lcMessageID)
ENDIF
CASE lcType = "CONFIRM"
lcMessageID = STREXTRACT(lcData, "<", ">", 2)
lnPacketNumber = VAL(STREXTRACT(lcData, "<", ">", 3))
SELECT SentMessages
SET ORDER TO MessageID
SEEK lcMessageID + STR(lnPacketNumber,5,0)
IF FOUND()
replace confirmed WITH .t.
ENDIF
SELECT RecvMessages
APPEND BLANK
replace type WITH lcType, ;
packet WITH lnPacketNumber, ;
MessageID WITH lcMessageID, ;
RawData WITH lcData
this.Parent.lastconfirm = lnPacketNumber
this.Parent.ConfirmationReceived(lcMessageID)
OTHERWISE
=MESSAGEBOX(lcData)
ENDCASE

Post by MikeA
Does anyone have sample code using winsock to transfer a file with a VFP
program clientsample.prg and serversample.prg? I'm still having problems
with no answers from prior posts and every time I think I'm that much
closer to a solution it just doesn't happen. I would think someone would
have some sample code somewhere. I searched the UT last night but found
nothing and I'm just getting a little tired of staying up til well after
3AM every night.
All help would be greatly appreciated.
Thanks,
Mike

Jeff Grippe

2010-01-05 20:50:56 UTC

I had similar problems which is what led me to develop my "message" wrapper.

I calculate the number of packets that I'll be sending and include a packet
number and count with each packet. The idea was that if I later found that I
was missing one, I could go back and request the missing packet.

As you found out, when you make the packets small enough, the winsock
control seems to get the job done accurately.

My senders were built so that they could be sending and receiving from
multiple other instances of the program at one time. A sender could initiate
a 40 packet send to Client A and a 50 packet send to Client B. As each
client sends back confirmation that they received the current packet, then
the sender sends the next packet.

I've never had to code the NAK. By using 100 byte messages, I've gotten
consistantly correct data. I think my message structure is too heavy for
sending large files, however. For a 100 byte message, I'm probably sending
between 30 and 50 bytes of wrapper.

Jeff

Post by MikeA
Jeff - thank you for your post.
My code works fine when I don't send too much information. It also works
if I do send information and use my own ACK and NAK. But, I should not
have to do that. My only conclusion at this time is Winsock has bugs in
the control itself causing problems when sending a lot of data.
But, I'm still researching all this.
Thanks,
Mike

Post by Jeff Grippe
This is a more complex problem that fits a newsgroup posting but I can
explain some concepts and provide some samples.
I have message types defined and a packet protocol that I use which may
be too "heavy" for sending large files. I break up what I'm sending into
packets of a fixed length. I send one packet and wait for a confirmation
before sending the next packet. Each instance of my application has an IP
address and port which it listens on. All applications can both send and
receive.
You need to bind the winsock control to some port. I have multiple users
each of whom binds to a separate port. I begin with port 1002 and look
for an available port. Once a user has bound to a port, I save the
results in a table. I use UDP Protocol and I save all of my "messages" in
a table. This isn't necessary but I like having the trail. What I call a
message is just a string of characters. There's no reason why it couldn't
be the contents of a file.
*** PORT BINDING
this.nIPPort = 1002 && first port to try binding to
this.oMSWinSock.protocol = 1 && UDP
this.oMSWinSock.Bind(this.nIPPort)
*** ERROR HANDLING IN CASE THE PORT DOESN'T BIND
LPARAMETERS nError, cMethod, nLine
DO CASE
CASE nError = 1429 && Problem binding port
this.nIPPort = this.nIPPort + 1 && try next sequential port
RETRY
OTHERWISE
=MESSAGEBOX(STR(nError,4,0)+" "+cMethod+" "+STR(nLine,4,0)+MESSAGE())
ENDCASE
*** SENDING DATA
this.oMSWinSock.RemoteHost = lcToIP && whatever ip address you are
sending to
this.oMSWinSock.RemotePort = lnToPort && whatever port you are sending to
*** My particular message protocol
lcFullMessage = "<MESSAGE>"+;
"<"+ALLTRIM(STR(lnPacket,5,0))+">"+;
"<"+ALLTRIM(STR(lnPacketCount,5,0))+">"+;
"<"+this.oMSWinSock.localip+">"+;
"<"+ALLTRIM(STR(this.nIPPort,6,0))+">"+;
"<"+pcmessageid+">"+;
"<"+SYS(2007,lcMessageSegment)+">"+;
"|"+lcMessageSegment
*** Remember that I save my messages
SELECT SentMessages
APPEND BLANK
REPLACE type WITH "MESSAGE", ;
packet WITH lnPacket, ;
packetcount WITH lnPacketCount, ;
ToIP WITH this.oMSWinSock.RemoteHost, ;
ToPort WITH this.oMSWinSock.RemotePort, ;
MessageID WITH pcMessageID, ;
CheckSum WITH SYS(2007, lcMessageSegment), ;
MessageData WITH lcMessageSegment, ;
RawData WITH lcFullMessage
*** SENDING THE MESSAGE
this.oMSWinSock.SendData(lcFullMessage)
The message comes in to the WinSock control and fires the
DataArrivalEvent. This in turn uses the GetData method to return the
data. That is where your code has to go to pickup the data that was sent.
*** RECEIVING THE DATA IN THE DataArrivalEvent
*** ActiveX Control Event ***
LPARAMETERS bytestotal
lcData = SPACE(256) && Define string to pass to GetData
* get fields and data
lcType = STREXTRACT(lcData, "<", ">", 1)
*** Processing the data using my protocol
DO CASE
CASE lcType = "MESSAGE" && message
lnSplitPos1 = AT("|", lcData, 1)
lnPacketNumber = VAL(STREXTRACT(lcData, "<", ">", 2))
lnTotalPackets = VAL(STREXTRACT(lcData, "<", ">", 3))
lcIPFrom = STREXTRACT(lcData, "<", ">", 4)
lnPortFrom = VAL(STREXTRACT(lcData, "<", ">", 5))
lcMessageID = STREXTRACT(lcData, "<", ">", 6)
lcCheckSum = STREXTRACT(lcData, "<", ">", 7)
lcMessage = SUBSTR(lcData, lnSplitPos1+1, LEN(lcData) - lnSplitPos1+1)
*** only necessary if you want to save the data like I do. Very useful
for debugging
SELECT RecvMessages
APPEND BLANK
replace type WITH lcType, ;
packet WITH lnPacketNumber, ;
packetcount WITH lnTotalPackets, ;
FromIP WITH lcIPFrom, ;
FromPort WITH lnPortFrom, ;
MessageID WITH lcMessageID, ;
CheckSum WITH lcCheckSum, ;
MessageData WITH lcMessage, ;
RawData WITH lcData
IF lcCheckSum <> SYS(2007, lcMessage) && checksum mismatch
=MESSAGEBOX("Check Sum Error" + lcData)
* send Re-Transmit Request - Not yet coded. This hasn't happened
yet (fortunately)
ELSE
* send Packet Confirmation
this.Parent.SendConfirmation(lcMessageID, lnPacketNumber)
ENDIF
IF lnPacketNumber = lnTotalPackets && last packet received
*** if all packets received correctly
this.Parent.Messagereceived(lcMessageID)
ENDIF
CASE lcType = "CONFIRM"
lcMessageID = STREXTRACT(lcData, "<", ">", 2)
lnPacketNumber = VAL(STREXTRACT(lcData, "<", ">", 3))
SELECT SentMessages
SET ORDER TO MessageID
SEEK lcMessageID + STR(lnPacketNumber,5,0)
IF FOUND()
replace confirmed WITH .t.
ENDIF
SELECT RecvMessages
APPEND BLANK
replace type WITH lcType, ;
packet WITH lnPacketNumber, ;
MessageID WITH lcMessageID, ;
RawData WITH lcData
this.Parent.lastconfirm = lnPacketNumber
this.Parent.ConfirmationReceived(lcMessageID)
OTHERWISE
=MESSAGEBOX(lcData)
ENDCASE

Post by MikeA
Does anyone have sample code using winsock to transfer a file with a VFP
program clientsample.prg and serversample.prg? I'm still having
problems with no answers from prior posts and every time I think I'm
that much closer to a solution it just doesn't happen. I would think
someone would have some sample code somewhere. I searched the UT last
night but found nothing and I'm just getting a little tired of staying
up til well after 3AM every night.
All help would be greatly appreciated.
Thanks,
Mike

MikeA

2010-01-06 06:23:01 UTC

When you are talking about a huge file transfer like 75 megs or several
hundred megs no matter how small the packets are it just does not work. I
think winsock is buggy and the only way to get it to work is to either use
nak/ack with large files. But, I'm just not understanding why. There is no
reason the socket should be buggy regardless of the size of the file because
all the validations are done at the TCP/IP layer so there is no reason we
should have to do any checksums at all.

Mike

Post by Jeff Grippe
I had similar problems which is what led me to develop my "message" wrapper.
I calculate the number of packets that I'll be sending and include a
packet number and count with each packet. The idea was that if I later
found that I was missing one, I could go back and request the missing
packet.
As you found out, when you make the packets small enough, the winsock
control seems to get the job done accurately.
My senders were built so that they could be sending and receiving from
multiple other instances of the program at one time. A sender could
initiate a 40 packet send to Client A and a 50 packet send to Client B. As
each client sends back confirmation that they received the current packet,
then the sender sends the next packet.
I've never had to code the NAK. By using 100 byte messages, I've gotten
consistantly correct data. I think my message structure is too heavy for
sending large files, however. For a 100 byte message, I'm probably sending
between 30 and 50 bytes of wrapper.
Jeff

Post by Jeff Grippe
This is a more complex problem that fits a newsgroup posting but I can
explain some concepts and provide some samples.
I have message types defined and a packet protocol that I use which may
be too "heavy" for sending large files. I break up what I'm sending into
packets of a fixed length. I send one packet and wait for a confirmation
before sending the next packet. Each instance of my application has an
IP address and port which it listens on. All applications can both send
and receive.
You need to bind the winsock control to some port. I have multiple users
each of whom binds to a separate port. I begin with port 1002 and look
for an available port. Once a user has bound to a port, I save the
results in a table. I use UDP Protocol and I save all of my "messages"
in a table. This isn't necessary but I like having the trail. What I
call a message is just a string of characters. There's no reason why it
couldn't be the contents of a file.
*** PORT BINDING
this.nIPPort = 1002 && first port to try binding to
this.oMSWinSock.protocol = 1 && UDP
this.oMSWinSock.Bind(this.nIPPort)
*** ERROR HANDLING IN CASE THE PORT DOESN'T BIND
LPARAMETERS nError, cMethod, nLine
DO CASE
CASE nError = 1429 && Problem binding port
this.nIPPort = this.nIPPort + 1 && try next sequential port
RETRY
OTHERWISE
=MESSAGEBOX(STR(nError,4,0)+" "+cMethod+" "+STR(nLine,4,0)+MESSAGE())
ENDCASE
*** SENDING DATA
this.oMSWinSock.RemoteHost = lcToIP && whatever ip address you
are sending to
this.oMSWinSock.RemotePort = lnToPort && whatever port you are sending to
*** My particular message protocol
lcFullMessage = "<MESSAGE>"+;
"<"+ALLTRIM(STR(lnPacket,5,0))+">"+;
"<"+ALLTRIM(STR(lnPacketCount,5,0))+">"+;
"<"+this.oMSWinSock.localip+">"+;
"<"+ALLTRIM(STR(this.nIPPort,6,0))+">"+;
"<"+pcmessageid+">"+;
"<"+SYS(2007,lcMessageSegment)+">"+;
"|"+lcMessageSegment
*** Remember that I save my messages
SELECT SentMessages
APPEND BLANK
REPLACE type WITH "MESSAGE", ;
packet WITH lnPacket, ;
packetcount WITH lnPacketCount, ;
ToIP WITH this.oMSWinSock.RemoteHost, ;
ToPort WITH this.oMSWinSock.RemotePort, ;
MessageID WITH pcMessageID, ;
CheckSum WITH SYS(2007, lcMessageSegment), ;
MessageData WITH lcMessageSegment, ;
RawData WITH lcFullMessage
*** SENDING THE MESSAGE
this.oMSWinSock.SendData(lcFullMessage)
The message comes in to the WinSock control and fires the
DataArrivalEvent. This in turn uses the GetData method to return the
data. That is where your code has to go to pickup the data that was sent.
*** RECEIVING THE DATA IN THE DataArrivalEvent
*** ActiveX Control Event ***
LPARAMETERS bytestotal
lcData = SPACE(256) && Define string to pass to GetData
* get fields and data
lcType = STREXTRACT(lcData, "<", ">", 1)
*** Processing the data using my protocol
DO CASE
CASE lcType = "MESSAGE" && message
lnSplitPos1 = AT("|", lcData, 1)
lnPacketNumber = VAL(STREXTRACT(lcData, "<", ">", 2))
lnTotalPackets = VAL(STREXTRACT(lcData, "<", ">", 3))
lcIPFrom = STREXTRACT(lcData, "<", ">", 4)
lnPortFrom = VAL(STREXTRACT(lcData, "<", ">", 5))
lcMessageID = STREXTRACT(lcData, "<", ">", 6)
lcCheckSum = STREXTRACT(lcData, "<", ">", 7)
lcMessage = SUBSTR(lcData, lnSplitPos1+1, LEN(lcData) -
lnSplitPos1+1)
*** only necessary if you want to save the data like I do. Very
useful for debugging
SELECT RecvMessages
APPEND BLANK
replace type WITH lcType, ;
packet WITH lnPacketNumber, ;
packetcount WITH lnTotalPackets, ;
FromIP WITH lcIPFrom, ;
FromPort WITH lnPortFrom, ;
MessageID WITH lcMessageID, ;
CheckSum WITH lcCheckSum, ;
MessageData WITH lcMessage, ;
RawData WITH lcData
IF lcCheckSum <> SYS(2007, lcMessage) && checksum mismatch
=MESSAGEBOX("Check Sum Error" + lcData)
* send Re-Transmit Request - Not yet coded. This hasn't happened
yet (fortunately)
ELSE
* send Packet Confirmation
this.Parent.SendConfirmation(lcMessageID, lnPacketNumber)
ENDIF
IF lnPacketNumber = lnTotalPackets && last packet received
*** if all packets received correctly
this.Parent.Messagereceived(lcMessageID)
ENDIF
CASE lcType = "CONFIRM"
lcMessageID = STREXTRACT(lcData, "<", ">", 2)
lnPacketNumber = VAL(STREXTRACT(lcData, "<", ">", 3))
SELECT SentMessages
SET ORDER TO MessageID
SEEK lcMessageID + STR(lnPacketNumber,5,0)
IF FOUND()
replace confirmed WITH .t.
ENDIF
SELECT RecvMessages
APPEND BLANK
replace type WITH lcType, ;
packet WITH lnPacketNumber, ;
MessageID WITH lcMessageID, ;
RawData WITH lcData
this.Parent.lastconfirm = lnPacketNumber
this.Parent.ConfirmationReceived(lcMessageID)
OTHERWISE
=MESSAGEBOX(lcData)
ENDCASE

Post by MikeA
Does anyone have sample code using winsock to transfer a file with a
VFP program clientsample.prg and serversample.prg? I'm still having
problems with no answers from prior posts and every time I think I'm
that much closer to a solution it just doesn't happen. I would think
someone would have some sample code somewhere. I searched the UT last
night but found nothing and I'm just getting a little tired of staying
up til well after 3AM every night.
All help would be greatly appreciated.
Thanks,
Mike

MikeA

2010-01-07 17:14:39 UTC

Does anyone have any ideas and/or solutions - I'm really getting kind of
desperate for a solution?

Thanks,
Mike

Post by MikeA
Does anyone have sample code using winsock to transfer a file with a VFP
program clientsample.prg and serversample.prg? I'm still having problems
with no answers from prior posts and every time I think I'm that much
closer to a solution it just doesn't happen. I would think someone would
have some sample code somewhere. I searched the UT last night but found
nothing and I'm just getting a little tired of staying up til well after
3AM every night.
All help would be greatly appreciated.
Thanks,
Mike

Richard Stecenko

2010-01-07 20:54:42 UTC

I've been playing around with Winsock since you made the first post
and I think that you are right when you suggest that Winsock won't
work for large files.

I could get about 20 to 30 megs of data going from the two forms
(client and server) in the Microsoft VFP sample. But as soon as I
started sending more, it just didn't work reliably.

I never could get the sendComplete to fire consistently. I couldn't
find any samples in VFP that did more than send very small chunks of
data. The VB samples I saw were also simple (i.e. trivial amounts of
data) or involved all sorts of extra acking and naking, which, you
suggest shouldn't be necessary). I did find one web site where the
developer claimed that Winsock will only work reliably in C. I read
more stuff on the Nagle Algorithm than I ever thought possible.

I'd say, give it up Mark. Find some other way to move those files.

Richard Stecenko
Interactive Computer Services Inc.
Winnipeg, Canada
204.453.2052

MikeA

2010-01-07 23:25:25 UTC

No - I can't give this up. First, I HAVE gotten it to work but only at half
speed by doing my own checksums.

I know that I can use some "heavy programming" to speed it up a bit but
TCP/IP should do all the checksums for me. At Catalyst.com (the control I'm
testing now and at least there is support unlike mabry which is currently
out) they claim it works fine for 100 meg files or more.

So - now I"m thinking that maybe the problem is from where the string gets
passed to the socket. Maybe I need to try another way to load the socket's
buffer. Perhaps ComArray or just a pointer to a spot in memory but I posted
that in another thread. Believe me, I've been banging my head against the
wall with this one and I can get it to work by doing my own checksums but
it's not as efficient speed wise or programming that should not be
necessary.

So I'm going to continue my search. But, I'm thinking maybe if I can pass
the string a different way then it might work. I can't use FTP or HTTP
sending or just a copy through a VPN because many of my clients may not have
any of that set up (in fact, most don't even have their own FTP and it
doesn't make sense to send it to a public FTP because then again it is going
to take twice as long and I can do that now. The simplest way is a socket.

As you can see - I don't give up easily (and never have) but I certainly
appreciate your support in this and I too disabled the Nagle and tried all
kinds of stuff. The only full success I had was when I created my own
checksums.

Please see my most recent thread.

Thank you again.
Mike

Post by Richard Stecenko
I've been playing around with Winsock since you made the first post
and I think that you are right when you suggest that Winsock won't
work for large files.
I could get about 20 to 30 megs of data going from the two forms
(client and server) in the Microsoft VFP sample. But as soon as I
started sending more, it just didn't work reliably.
I never could get the sendComplete to fire consistently. I couldn't
find any samples in VFP that did more than send very small chunks of
data. The VB samples I saw were also simple (i.e. trivial amounts of
data) or involved all sorts of extra acking and naking, which, you
suggest shouldn't be necessary). I did find one web site where the
developer claimed that Winsock will only work reliably in C. I read
more stuff on the Nagle Algorithm than I ever thought possible.
I'd say, give it up Mark. Find some other way to move those files.
Richard Stecenko
Interactive Computer Services Inc.
Winnipeg, Canada
204.453.2052

MikeA

2010-01-07 23:26:42 UTC

Richard - I almost forgot - in addition to my prior post - I know that
winsock is not reliable but I'm using third party sockets and they too are
not working. I know for a fact that mabry is way more stable than winsock
but again, it too has problems with file transfer.

Mike

Richard Stecenko

2010-01-08 02:16:00 UTC

I know that winsock is not reliable
but I'm using third party sockets and they too are not working.

OK, so if Winsock is not reliable, and your third party sockets are
not reliable, what's left over?

From what I've read about the Nagle algorithm, you have two choices:
you can have a fast transfer or a reliable transfer. You just can't
have both. Again, from what I've read, it is possible to just jam in
too much data. That's why that one web site claimed that programs that
use Winsock should be written in C.

Now, don't take what I'm saying at face value or as gospel. I was
curious about what you are trying to do and read a few things and
tried a few things. But, (I hate to say this), your problem is not my
problem. And if I were you, I'd try another approach.

Richard Stecenko
Interactive Computer Services Inc.
Winnipeg, Canada
204.453.2052

MikeA

2010-01-08 08:30:08 UTC

Thank you again for your continued support. If you have a better approach
to this then I'm all ears. But, here are my thoughts. First, whether Nagle
is enabled or not, File Transfer through the socket still does not work
(without implementing your own checksums). But, like I say, I can get it to
work and have by using my own checksums. So, the socket solution will work
it's just I want the code as "clean" as possible without sacrificing speed.

A socket is the most simple "out of the box" solution. For example,
programs like pc Anywhere or gotomypc, etc. do not force the user to have
their own FTP or HTTP when you want to utilize them to do a file transfer
from or to the host. I don't want my customers to have to be dependent on
some additional setup such as FTP servers because this is a major piece I am
adding to my Foxpro database application.

As far as what I'm trying to do - it goes like this. My clients right now
have the ability to copy my entire database application to their laptop
computers then work "offline" without any internet or network. Then they
can take their laptop computer back to their site and sync the data. I want
to allow them to sync from anywhere via the internet in as few steps as
possible. The ability for them to do this is huge. Also, a socket is ideal
for this. We are talking about several steps internally here so for
example, client may need to transfer a file to the server, server then
processes that file then server sends client a few files and client
processes those and so on back and forth in a very ordered fashion until the
sync is done. The files must be processed in a specific order and the
client server approach is idea. But again, I'm already doing it with
checksums it's just the programming gets very messy and twice as bad if I
try and optimize the speed programmatically and before I dedicate a lot of
time to trying to optimize the code I would like to find a way to do it
RIGHT where TCP/IP does all the checksums for me so as to have the speed
optimized.

I agree about it being my problem and this forum is great as we are all here
to help each other but unless one comes up with a better solution to what
I'm trying to do - I'm going to do my best to optimize the socket as a
solution.

On another note, honestly, it's hard for me to believe that for all the time
VFP has been around, something as basic as a "file transfer" using sockets
with VFP has not been posted before. I know that several threads have been
written on how to do messaging in Visual FoxPro and again sockets are an
ideal solution and work great. I'm just trying to extend that a little
further to a file transfer.

Thanks again,
Mike

I know that winsock is not reliable
but I'm using third party sockets and they too are not working.

OK, so if Winsock is not reliable, and your third party sockets are
not reliable, what's left over?
you can have a fast transfer or a reliable transfer. You just can't
have both. Again, from what I've read, it is possible to just jam in
too much data. That's why that one web site claimed that programs that
use Winsock should be written in C.
Now, don't take what I'm saying at face value or as gospel. I was
curious about what you are trying to do and read a few things and
tried a few things. But, (I hate to say this), your problem is not my
problem. And if I were you, I'd try another approach.
Richard Stecenko
Interactive Computer Services Inc.
Winnipeg, Canada
204.453.2052

Jeff Grippe

2010-01-11 21:47:52 UTC

A simple checksum shouldn't slow the transfer down that much. I designed a
more verbose messaging package because I knew my messages would be short and
I wanted a richer package.

If you break the file into packets and use a lighter weight protocol in
which you have 100 bytes (or a little more) of message and only a few bytes
of checksum, then you should get decent performance. You aren't going to see
anywhere near the performance that you would get from something like a
mapped drive no matter what you do. But it should work well enough with your
basic ACK/NAK that you've setup.

FWIW, I've run into the same limitations that you have and I don't have a
better solution.

Jeff

Post by MikeA
Thank you again for your continued support. If you have a better approach
to this then I'm all ears. But, here are my thoughts. First, whether
Nagle is enabled or not, File Transfer through the socket still does not
work (without implementing your own checksums). But, like I say, I can
get it to work and have by using my own checksums. So, the socket
solution will work it's just I want the code as "clean" as possible
without sacrificing speed.
A socket is the most simple "out of the box" solution. For example,
programs like pc Anywhere or gotomypc, etc. do not force the user to have
their own FTP or HTTP when you want to utilize them to do a file transfer
from or to the host. I don't want my customers to have to be dependent on
some additional setup such as FTP servers because this is a major piece I
am adding to my Foxpro database application.
As far as what I'm trying to do - it goes like this. My clients right now
have the ability to copy my entire database application to their laptop
computers then work "offline" without any internet or network. Then they
can take their laptop computer back to their site and sync the data. I
want to allow them to sync from anywhere via the internet in as few steps
as possible. The ability for them to do this is huge. Also, a socket is
ideal for this. We are talking about several steps internally here so for
example, client may need to transfer a file to the server, server then
processes that file then server sends client a few files and client
processes those and so on back and forth in a very ordered fashion until
the sync is done. The files must be processed in a specific order and the
client server approach is idea. But again, I'm already doing it with
checksums it's just the programming gets very messy and twice as bad if I
try and optimize the speed programmatically and before I dedicate a lot of
time to trying to optimize the code I would like to find a way to do it
RIGHT where TCP/IP does all the checksums for me so as to have the speed
optimized.
I agree about it being my problem and this forum is great as we are all
here to help each other but unless one comes up with a better solution to
what I'm trying to do - I'm going to do my best to optimize the socket as
a solution.
On another note, honestly, it's hard for me to believe that for all the
time VFP has been around, something as basic as a "file transfer" using
sockets with VFP has not been posted before. I know that several threads
have been written on how to do messaging in Visual FoxPro and again
sockets are an ideal solution and work great. I'm just trying to extend
that a little further to a file transfer.
Thanks again,
Mike

I know that winsock is not reliable
but I'm using third party sockets and they too are not working.

MikeA

2010-01-11 22:31:04 UTC

The problem I think boils down to strategy and programming algorithms. If I
can't just flood the socket with data and let TCP/IP do the work for me of
verifying the data then it seems like I have to have another strategy for
continuously keeping data flowing along the socket otherwise it will slow to
a crawl. For example, the worst strategy would be to:

Send Packet A to server
Have server verify checksum
Send checksum back to client
Have client resend if checksum is bad

That would be terrible because the server would not be continuing to receive
data. It's really lousy that I can't just let TCP/IP verify all this for
me.

What other strategies have you used to make file transfer work for you?
It's just hard to imagine having to resend thousands of packets due to
checksum errors.

Thanks,
Mike

Post by Jeff Grippe
A simple checksum shouldn't slow the transfer down that much. I designed a
more verbose messaging package because I knew my messages would be short
and I wanted a richer package.
If you break the file into packets and use a lighter weight protocol in
which you have 100 bytes (or a little more) of message and only a few
bytes of checksum, then you should get decent performance. You aren't
going to see anywhere near the performance that you would get from
something like a mapped drive no matter what you do. But it should work
well enough with your basic ACK/NAK that you've setup.
FWIW, I've run into the same limitations that you have and I don't have a
better solution.
Jeff

Post by MikeA
Thank you again for your continued support. If you have a better
approach to this then I'm all ears. But, here are my thoughts. First,
whether Nagle is enabled or not, File Transfer through the socket still
does not work (without implementing your own checksums). But, like I
say, I can get it to work and have by using my own checksums. So, the
socket solution will work it's just I want the code as "clean" as
possible without sacrificing speed.
A socket is the most simple "out of the box" solution. For example,
programs like pc Anywhere or gotomypc, etc. do not force the user to have
their own FTP or HTTP when you want to utilize them to do a file transfer
from or to the host. I don't want my customers to have to be dependent
on some additional setup such as FTP servers because this is a major
piece I am adding to my Foxpro database application.
As far as what I'm trying to do - it goes like this. My clients right
now have the ability to copy my entire database application to their
laptop computers then work "offline" without any internet or network.
Then they can take their laptop computer back to their site and sync the
data. I want to allow them to sync from anywhere via the internet in as
few steps as possible. The ability for them to do this is huge. Also, a
socket is ideal for this. We are talking about several steps internally
here so for example, client may need to transfer a file to the server,
server then processes that file then server sends client a few files and
client processes those and so on back and forth in a very ordered fashion
until the sync is done. The files must be processed in a specific order
and the client server approach is idea. But again, I'm already doing it
with checksums it's just the programming gets very messy and twice as bad
if I try and optimize the speed programmatically and before I dedicate a
lot of time to trying to optimize the code I would like to find a way to
do it RIGHT where TCP/IP does all the checksums for me so as to have the
speed optimized.
I agree about it being my problem and this forum is great as we are all
here to help each other but unless one comes up with a better solution to
what I'm trying to do - I'm going to do my best to optimize the socket as
a solution.
On another note, honestly, it's hard for me to believe that for all the
time VFP has been around, something as basic as a "file transfer" using
sockets with VFP has not been posted before. I know that several threads
have been written on how to do messaging in Visual FoxPro and again
sockets are an ideal solution and work great. I'm just trying to extend
that a little further to a file transfer.
Thanks again,
Mike

I know that winsock is not reliable
but I'm using third party sockets and they too are not working.

MikeA

2010-01-11 22:43:33 UTC

Just to elaborate on this a little further:

If I use the strategy of flooding the socket and tracking checksums at the
server then it is a lot of heavy programming that it seems like I should not
have to do since TCP/IP already does this for me.

We are talking about something like this as an example:

Server receives 1000 packets and say 50 are bad
Server request to resend those 50 packets from the client
Server now needs to rebuild the file to merge those 50 packets where they
belong in the file (this requires a re-read of the file from the client and
rewrite of the file at the server)
Server needs to repeat the steps if of the 50 packets some of those are bad
and continue repeating until all the checksums are good.

This just seems like a lot of heavy programming that should not be required
if TCP/IP is already doing the checksums.

Mike

Post by MikeA
The problem I think boils down to strategy and programming algorithms. If
I can't just flood the socket with data and let TCP/IP do the work for me
of verifying the data then it seems like I have to have another strategy
for continuously keeping data flowing along the socket otherwise it will
Send Packet A to server
Have server verify checksum
Send checksum back to client
Have client resend if checksum is bad
That would be terrible because the server would not be continuing to
receive data. It's really lousy that I can't just let TCP/IP verify all
this for me.
What other strategies have you used to make file transfer work for you?
It's just hard to imagine having to resend thousands of packets due to
checksum errors.
Thanks,
Mike

Post by MikeA
Thank you again for your continued support. If you have a better
approach to this then I'm all ears. But, here are my thoughts. First,
whether Nagle is enabled or not, File Transfer through the socket still
does not work (without implementing your own checksums). But, like I
say, I can get it to work and have by using my own checksums. So, the
socket solution will work it's just I want the code as "clean" as
possible without sacrificing speed.
A socket is the most simple "out of the box" solution. For example,
programs like pc Anywhere or gotomypc, etc. do not force the user to
have their own FTP or HTTP when you want to utilize them to do a file
transfer from or to the host. I don't want my customers to have to be
dependent on some additional setup such as FTP servers because this is a
major piece I am adding to my Foxpro database application.
As far as what I'm trying to do - it goes like this. My clients right
now have the ability to copy my entire database application to their
laptop computers then work "offline" without any internet or network.
Then they can take their laptop computer back to their site and sync the
data. I want to allow them to sync from anywhere via the internet in as
few steps as possible. The ability for them to do this is huge. Also, a
socket is ideal for this. We are talking about several steps internally
here so for example, client may need to transfer a file to the server,
server then processes that file then server sends client a few files and
client processes those and so on back and forth in a very ordered
fashion until the sync is done. The files must be processed in a
specific order and the client server approach is idea. But again, I'm
already doing it with checksums it's just the programming gets very
messy and twice as bad if I try and optimize the speed programmatically
and before I dedicate a lot of time to trying to optimize the code I
would like to find a way to do it RIGHT where TCP/IP does all the
checksums for me so as to have the speed optimized.
I agree about it being my problem and this forum is great as we are all
here to help each other but unless one comes up with a better solution
to what I'm trying to do - I'm going to do my best to optimize the
socket as a solution.
On another note, honestly, it's hard for me to believe that for all the
time VFP has been around, something as basic as a "file transfer" using
sockets with VFP has not been posted before. I know that several
threads have been written on how to do messaging in Visual FoxPro and
again sockets are an ideal solution and work great. I'm just trying to
extend that a little further to a file transfer.
Thanks again,
Mike

I know that winsock is not reliable
but I'm using third party sockets and they too are not working.

MikeA

2010-01-13 06:25:53 UTC

Jeff - I have a question - why break the file into 100 byte packtes to send
instead of say 1000 bytes with a checksum? Then I'm sending one checksum
rather than 10. Any thoughts on this? Also, I'm trying to experiment with
the idea of possibly getting a false positive with a checksum but I"m not
sure how likely that is but then again the more packets that are sent the
more likely that I could get a false positive. That alone would seem to
mean the odds of a false positive are 10 times greater using 100 byte
packets rather than a 1000 byte packet.

Thanks again for all insights.

Mike

Post by MikeA
Thank you again for your continued support. If you have a better
approach to this then I'm all ears. But, here are my thoughts. First,
whether Nagle is enabled or not, File Transfer through the socket still
does not work (without implementing your own checksums). But, like I
say, I can get it to work and have by using my own checksums. So, the
socket solution will work it's just I want the code as "clean" as
possible without sacrificing speed.
A socket is the most simple "out of the box" solution. For example,
programs like pc Anywhere or gotomypc, etc. do not force the user to have
their own FTP or HTTP when you want to utilize them to do a file transfer
from or to the host. I don't want my customers to have to be dependent
on some additional setup such as FTP servers because this is a major
piece I am adding to my Foxpro database application.
As far as what I'm trying to do - it goes like this. My clients right
now have the ability to copy my entire database application to their
laptop computers then work "offline" without any internet or network.
Then they can take their laptop computer back to their site and sync the
data. I want to allow them to sync from anywhere via the internet in as
few steps as possible. The ability for them to do this is huge. Also, a
socket is ideal for this. We are talking about several steps internally
here so for example, client may need to transfer a file to the server,
server then processes that file then server sends client a few files and
client processes those and so on back and forth in a very ordered fashion
until the sync is done. The files must be processed in a specific order
and the client server approach is idea. But again, I'm already doing it
with checksums it's just the programming gets very messy and twice as bad
if I try and optimize the speed programmatically and before I dedicate a
lot of time to trying to optimize the code I would like to find a way to
do it RIGHT where TCP/IP does all the checksums for me so as to have the
speed optimized.
I agree about it being my problem and this forum is great as we are all
here to help each other but unless one comes up with a better solution to
what I'm trying to do - I'm going to do my best to optimize the socket as
a solution.
On another note, honestly, it's hard for me to believe that for all the
time VFP has been around, something as basic as a "file transfer" using
sockets with VFP has not been posted before. I know that several threads
have been written on how to do messaging in Visual FoxPro and again
sockets are an ideal solution and work great. I'm just trying to extend
that a little further to a file transfer.
Thanks again,
Mike

I know that winsock is not reliable
but I'm using third party sockets and they too are not working.

Jeff Grippe

2010-01-13 18:43:02 UTC

I, like you, found that WinSock was not reliable for large amounts of data.
My desicion to use 100 byte packages was somewhat random although I wanted
the ability to compare what the sender and receiver were getting and going
much over 100 bytes would have made visual inspection difficult.

There is no reason not to use 1000 byte packets once you've convinced
yourself that you will get reliable performance with that size.

Since my messages are stored in a database, I am now sending very small
messages which only serve to let the receiver know that a database record
has been updated or created. I used to send the contents of the database
record (rtf data) until I realized that I could just as easily put the data
into a memo field and simply send a message telling the receiver where to
find the data.

My messages are now all well under 100 bytes so I had no need to go larger.

I don't know that you need a dedicated FTP server to use FTP for file
transfer. I think that various tools that are available let you do both
sending and receiving via FTP. I'm not sure, however, as I've only uploaded
to a server and pulled down from a server using FTP.

Jeff

Post by MikeA
Jeff - I have a question - why break the file into 100 byte packtes to
send instead of say 1000 bytes with a checksum? Then I'm sending one
checksum rather than 10. Any thoughts on this? Also, I'm trying to
experiment with the idea of possibly getting a false positive with a
checksum but I"m not sure how likely that is but then again the more
packets that are sent the more likely that I could get a false positive.
That alone would seem to mean the odds of a false positive are 10 times
greater using 100 byte packets rather than a 1000 byte packet.
Thanks again for all insights.
Mike

Post by MikeA
Thank you again for your continued support. If you have a better
approach to this then I'm all ears. But, here are my thoughts. First,
whether Nagle is enabled or not, File Transfer through the socket still
does not work (without implementing your own checksums). But, like I
say, I can get it to work and have by using my own checksums. So, the
socket solution will work it's just I want the code as "clean" as
possible without sacrificing speed.
A socket is the most simple "out of the box" solution. For example,
programs like pc Anywhere or gotomypc, etc. do not force the user to
have their own FTP or HTTP when you want to utilize them to do a file
transfer from or to the host. I don't want my customers to have to be
dependent on some additional setup such as FTP servers because this is a
major piece I am adding to my Foxpro database application.
As far as what I'm trying to do - it goes like this. My clients right
now have the ability to copy my entire database application to their
laptop computers then work "offline" without any internet or network.
Then they can take their laptop computer back to their site and sync the
data. I want to allow them to sync from anywhere via the internet in as
few steps as possible. The ability for them to do this is huge. Also, a
socket is ideal for this. We are talking about several steps internally
here so for example, client may need to transfer a file to the server,
server then processes that file then server sends client a few files and
client processes those and so on back and forth in a very ordered
fashion until the sync is done. The files must be processed in a
specific order and the client server approach is idea. But again, I'm
already doing it with checksums it's just the programming gets very
messy and twice as bad if I try and optimize the speed programmatically
and before I dedicate a lot of time to trying to optimize the code I
would like to find a way to do it RIGHT where TCP/IP does all the
checksums for me so as to have the speed optimized.
I agree about it being my problem and this forum is great as we are all
here to help each other but unless one comes up with a better solution
to what I'm trying to do - I'm going to do my best to optimize the
socket as a solution.
On another note, honestly, it's hard for me to believe that for all the
time VFP has been around, something as basic as a "file transfer" using
sockets with VFP has not been posted before. I know that several
threads have been written on how to do messaging in Visual FoxPro and
again sockets are an ideal solution and work great. I'm just trying to
extend that a little further to a file transfer.
Thanks again,
Mike

I know that winsock is not reliable
but I'm using third party sockets and they too are not working.

MikeA

2010-01-14 05:23:19 UTC

I don't know if it is possible to do FTP from just "computer to computer"
without an FTP server. Also, I realized thinking further about the solution
to creating my own checksums can still cause problems.

Suppose CLIENT sends 10,000 packets of 1000 bytes to SERVER (or 10 megs).
Suppose SERVER says 100 packets are bad. SERVER now needs to send CLIENT
the packet numbers that are bad so that CLIENT can resend them. So, that
would be 100 integers to identify the packet numbers that need to be resent.
Each integer may need to be identified by four bytes or 4 * 100 = 400 bytes
that need to be sent to the CLIENT to specify which packets need to be
resent. Now, the SERVER could in theory send 400 bytes to CLIENT where
some of the bytes are corrupted. In other words, the "resend information"
could be corrupted.

What a mess....uggg.

Mike

Post by Jeff Grippe
I, like you, found that WinSock was not reliable for large amounts of
data. My desicion to use 100 byte packages was somewhat random although I
wanted the ability to compare what the sender and receiver were getting
and going much over 100 bytes would have made visual inspection difficult.
There is no reason not to use 1000 byte packets once you've convinced
yourself that you will get reliable performance with that size.
Since my messages are stored in a database, I am now sending very small
messages which only serve to let the receiver know that a database record
has been updated or created. I used to send the contents of the database
record (rtf data) until I realized that I could just as easily put the
data into a memo field and simply send a message telling the receiver
where to find the data.
My messages are now all well under 100 bytes so I had no need to go larger.
I don't know that you need a dedicated FTP server to use FTP for file
transfer. I think that various tools that are available let you do both
sending and receiving via FTP. I'm not sure, however, as I've only
uploaded to a server and pulled down from a server using FTP.
Jeff

Post by Jeff Grippe
A simple checksum shouldn't slow the transfer down that much. I designed
a more verbose messaging package because I knew my messages would be
short and I wanted a richer package.
If you break the file into packets and use a lighter weight protocol in
which you have 100 bytes (or a little more) of message and only a few
bytes of checksum, then you should get decent performance. You aren't
going to see anywhere near the performance that you would get from
something like a mapped drive no matter what you do. But it should work
well enough with your basic ACK/NAK that you've setup.
FWIW, I've run into the same limitations that you have and I don't have
a better solution.
Jeff

Post by MikeA
Thank you again for your continued support. If you have a better
approach to this then I'm all ears. But, here are my thoughts. First,
whether Nagle is enabled or not, File Transfer through the socket still
does not work (without implementing your own checksums). But, like I
say, I can get it to work and have by using my own checksums. So, the
socket solution will work it's just I want the code as "clean" as
possible without sacrificing speed.
A socket is the most simple "out of the box" solution. For example,
programs like pc Anywhere or gotomypc, etc. do not force the user to
have their own FTP or HTTP when you want to utilize them to do a file
transfer from or to the host. I don't want my customers to have to be
dependent on some additional setup such as FTP servers because this is
a major piece I am adding to my Foxpro database application.
As far as what I'm trying to do - it goes like this. My clients right
now have the ability to copy my entire database application to their
laptop computers then work "offline" without any internet or network.
Then they can take their laptop computer back to their site and sync
the data. I want to allow them to sync from anywhere via the internet
in as few steps as possible. The ability for them to do this is huge.
Also, a socket is ideal for this. We are talking about several steps
internally here so for example, client may need to transfer a file to
the server, server then processes that file then server sends client a
few files and client processes those and so on back and forth in a very
ordered fashion until the sync is done. The files must be processed in
a specific order and the client server approach is idea. But again,
I'm already doing it with checksums it's just the programming gets very
messy and twice as bad if I try and optimize the speed programmatically
and before I dedicate a lot of time to trying to optimize the code I
would like to find a way to do it RIGHT where TCP/IP does all the
checksums for me so as to have the speed optimized.
I agree about it being my problem and this forum is great as we are all
here to help each other but unless one comes up with a better solution
to what I'm trying to do - I'm going to do my best to optimize the
socket as a solution.
On another note, honestly, it's hard for me to believe that for all the
time VFP has been around, something as basic as a "file transfer" using
sockets with VFP has not been posted before. I know that several
threads have been written on how to do messaging in Visual FoxPro and
again sockets are an ideal solution and work great. I'm just trying to
extend that a little further to a file transfer.
Thanks again,
Mike

I know that winsock is not reliable
but I'm using third party sockets and they too are not working.

MikeA

2010-01-14 05:23:57 UTC

Jeff - I'm finding the third party controls (at least two) also are not
reliable for large amounts of data. Is there a third party control that IS
reliable that I can use?

Thanks,
Mike

Post by Jeff Grippe
A simple checksum shouldn't slow the transfer down that much. I designed
a more verbose messaging package because I knew my messages would be
short and I wanted a richer package.
If you break the file into packets and use a lighter weight protocol in
which you have 100 bytes (or a little more) of message and only a few
bytes of checksum, then you should get decent performance. You aren't
going to see anywhere near the performance that you would get from
something like a mapped drive no matter what you do. But it should work
well enough with your basic ACK/NAK that you've setup.
FWIW, I've run into the same limitations that you have and I don't have
a better solution.
Jeff

Post by MikeA
Thank you again for your continued support. If you have a better
approach to this then I'm all ears. But, here are my thoughts. First,
whether Nagle is enabled or not, File Transfer through the socket still
does not work (without implementing your own checksums). But, like I
say, I can get it to work and have by using my own checksums. So, the
socket solution will work it's just I want the code as "clean" as
possible without sacrificing speed.
A socket is the most simple "out of the box" solution. For example,
programs like pc Anywhere or gotomypc, etc. do not force the user to
have their own FTP or HTTP when you want to utilize them to do a file
transfer from or to the host. I don't want my customers to have to be
dependent on some additional setup such as FTP servers because this is
a major piece I am adding to my Foxpro database application.
As far as what I'm trying to do - it goes like this. My clients right
now have the ability to copy my entire database application to their
laptop computers then work "offline" without any internet or network.
Then they can take their laptop computer back to their site and sync
the data. I want to allow them to sync from anywhere via the internet
in as few steps as possible. The ability for them to do this is huge.
Also, a socket is ideal for this. We are talking about several steps
internally here so for example, client may need to transfer a file to
the server, server then processes that file then server sends client a
few files and client processes those and so on back and forth in a very
ordered fashion until the sync is done. The files must be processed in
a specific order and the client server approach is idea. But again,
I'm already doing it with checksums it's just the programming gets very
messy and twice as bad if I try and optimize the speed programmatically
and before I dedicate a lot of time to trying to optimize the code I
would like to find a way to do it RIGHT where TCP/IP does all the
checksums for me so as to have the speed optimized.
I agree about it being my problem and this forum is great as we are all
here to help each other but unless one comes up with a better solution
to what I'm trying to do - I'm going to do my best to optimize the
socket as a solution.
On another note, honestly, it's hard for me to believe that for all the
time VFP has been around, something as basic as a "file transfer" using
sockets with VFP has not been posted before. I know that several
threads have been written on how to do messaging in Visual FoxPro and
again sockets are an ideal solution and work great. I'm just trying to
extend that a little further to a file transfer.
Thanks again,
Mike

I know that winsock is not reliable
but I'm using third party sockets and they too are not working.

Man-wai Chang to The Door (24000bps)

2010-01-08 10:23:53 UTC

Post by MikeA
Does anyone have any ideas and/or solutions - I'm really getting kind of
desperate for a solution?

What if you check the errors yourself rather than relying on the stack?

--
@~@ Might, Courage, Vision, SINCERITY.
/ v \ Simplicity is Beauty! May the Force and Farce be with you!
/( _ )\ (x86_64 Ubuntu 9.10) Linux 2.6.32.2
^ ^ 18:23:01 up 18 days 1:37 2 users load average: 1.03 1.01 1.00
不借貸! 不詐騙! 不援交! 不打交! 不打劫! 不自殺! 請考慮綜援 (CSSA):
http://www.swd.gov.hk/tc/index/site_pubsvc/page_socsecu/sub_addressesa

MikeA

2010-01-09 02:09:06 UTC

I'm able to check them myself and I have programmatically done that with
checksums but the problem is it slows the transfer rate way down and becomes
a heavy programming job for what should be just a few lines of code. The
vendor that supplies this sockets says I have to pass it a Byte Array like a
Visual Basic SAFEARRAY in variant (whatever that means). Is there a way to
do this in Visual FoxPro or perhaps an ActiveX control to allow me to
convert a string to this.

Thanks,
Mike

Post by MikeA
Does anyone have any ideas and/or solutions - I'm really getting kind of
desperate for a solution?

What if you check the errors yourself rather than relying on the stack?
--
@~@ Might, Courage, Vision, SINCERITY.
/ v \ Simplicity is Beauty! May the Force and Farce be with you!
/( _ )\ (x86_64 Ubuntu 9.10) Linux 2.6.32.2
^ ^ 18:23:01 up 18 days 1:37 2 users load average: 1.03 1.01 1.00
http://www.swd.gov.hk/tc/index/site_pubsvc/page_socsecu/sub_addressesa

Man-wai Chang to The Door (24000bps)

2010-01-09 08:19:55 UTC

If it worked, then use it until there was a better solution. :)

Post by MikeA
The vendor that supplies this sockets says I have to pass it a Byte
Array like a Visual Basic SAFEARRAY in variant (whatever that means).
Is there a way to do this in Visual FoxPro or perhaps an
ActiveX control to allow me to convert a string to this.

I thought you were using the standard M$ WinSock ActiveX control. Seems
that you were not....

--
@~@ Might, Courage, Vision, SINCERITY.
/ v \ Simplicity is Beauty! May the Force and Farce be with you!
/( _ )\ (x86_64 Ubuntu 9.10) Linux 2.6.32.3
^ ^ 16:18:01 up 1:31 0 users load average: 1.00 1.00 1.00
不借貸! 不詐騙! 不援交! 不打交! 不打劫! 不自殺! 請考慮綜援 (CSSA):
http://www.swd.gov.hk/tc/index/site_pubsvc/page_socsecu/sub_addressesa

Jeff Grippe

2010-01-12 18:13:17 UTC

Have you considered using FTP which was designed for file transfer?

There are several good libraries for FTP such as WW Internet tools from West
Wind. I'm sure there are free ones too. That should give you the performance
you want and the stability that you need.

Jeff

Post by MikeA
I'm able to check them myself and I have programmatically done that with
checksums but the problem is it slows the transfer rate way down and
becomes a heavy programming job for what should be just a few lines of
code. The vendor that supplies this sockets says I have to pass it a Byte
Array like a Visual Basic SAFEARRAY in variant (whatever that means). Is
there a way to do this in Visual FoxPro or perhaps an ActiveX control to
allow me to convert a string to this.
Thanks,
Mike

Post by MikeA
Does anyone have any ideas and/or solutions - I'm really getting kind of
desperate for a solution?

MikeA

2010-01-13 00:19:54 UTC

I have considered it but the problem is a lot of my users do not have an FTP
server.

On another note - the vendor that provides the socket I'm testing says that
it works fine in VB with the SafeArray. So, now I'm thinking maybe there is
an ActiveX control I can use to populate the socket's buffer.

Otherwise, I'll just have to use the sockets and do my own checksums. It's
really a waste because I would love to just do my own checksums and then
just turn of the TCP/IP checksums otherwise it's just a lot of unnecessary
data slowing down the transfer.

Thanks,
Mike

Post by Jeff Grippe
Have you considered using FTP which was designed for file transfer?
There are several good libraries for FTP such as WW Internet tools from
West Wind. I'm sure there are free ones too. That should give you the
performance you want and the stability that you need.
Jeff

Post by MikeA
I'm able to check them myself and I have programmatically done that with
checksums but the problem is it slows the transfer rate way down and
becomes a heavy programming job for what should be just a few lines of
code. The vendor that supplies this sockets says I have to pass it a
Byte Array like a Visual Basic SAFEARRAY in variant (whatever that
means). Is there a way to do this in Visual FoxPro or perhaps an ActiveX
control to allow me to convert a string to this.
Thanks,
Mike

Post by MikeA
Does anyone have any ideas and/or solutions - I'm really getting kind of
desperate for a solution?

2010-01-13 07:13:11 UTC

Hi Mike,

Why don't you write it in VB using SafeArray to see if it really works. If
it does, create a wrapper for it in FoxPro. On the flip side, it will run
out of process.

If you're writing this kind of code, VB should be a piece of cake even if
you don't know VB. Plus, you probably have VB sample code from the vendor.

Good Luck,
ED.

"MikeA" <***@appellsoftware.com> wrote in message news:eWAzbY%***@TK2MSFTNGP05.phx.gbl...
I have considered it but the problem is a lot of my users do not have an FTP
server.

On another note - the vendor that provides the socket I'm testing says that
it works fine in VB with the SafeArray. So, now I'm thinking maybe there is
an ActiveX control I can use to populate the socket's buffer.

Otherwise, I'll just have to use the sockets and do my own checksums. It's
really a waste because I would love to just do my own checksums and then
just turn of the TCP/IP checksums otherwise it's just a lot of unnecessary
data slowing down the transfer.

Thanks,
Mike

Post by MikeA
I'm able to check them myself and I have programmatically done that with
checksums but the problem is it slows the transfer rate way down and
becomes a heavy programming job for what should be just a few lines of
code. The vendor that supplies this sockets says I have to pass it a
Byte Array like a Visual Basic SAFEARRAY in variant (whatever that
means). Is there a way to do this in Visual FoxPro or perhaps an ActiveX
control to allow me to convert a string to this.
Thanks,
Mike

Post by MikeA
Does anyone have any ideas and/or solutions - I'm really getting kind of
desperate for a solution?

MikeA

2010-01-13 08:47:07 UTC

True but I don't know how to wrap the VB to VFP

Mike

Post by Richard Stecenko
Hi Mike,
Why don't you write it in VB using SafeArray to see if it really works. If
it does, create a wrapper for it in FoxPro. On the flip side, it will run
out of process.
If you're writing this kind of code, VB should be a piece of cake even if
you don't know VB. Plus, you probably have VB sample code from the vendor.
Good Luck,
ED.
I have considered it but the problem is a lot of my users do not have an FTP
server.
On another note - the vendor that provides the socket I'm testing says that
it works fine in VB with the SafeArray. So, now I'm thinking maybe there is
an ActiveX control I can use to populate the socket's buffer.
Otherwise, I'll just have to use the sockets and do my own checksums.
It's
really a waste because I would love to just do my own checksums and then
just turn of the TCP/IP checksums otherwise it's just a lot of unnecessary
data slowing down the transfer.
Thanks,
Mike

Post by MikeA
I'm able to check them myself and I have programmatically done that with
checksums but the problem is it slows the transfer rate way down and
becomes a heavy programming job for what should be just a few lines of
code. The vendor that supplies this sockets says I have to pass it a
Byte Array like a Visual Basic SAFEARRAY in variant (whatever that
means). Is there a way to do this in Visual FoxPro or perhaps an ActiveX
control to allow me to convert a string to this.
Thanks,
Mike

Post by MikeA
Does anyone have any ideas and/or solutions - I'm really getting kind of
desperate for a solution?

Man-wai Chang to The Door (24000bps)

2010-01-13 09:24:34 UTC

Post by MikeA
True but I don't know how to wrap the VB to VFP

VB could produce a standard DLL. Just make sure that the DLL doesn't
have user interface.

--
@~@ Might, Courage, Vision, SINCERITY.
/ v \ Simplicity is Beauty! May the Force and Farce be with you!
/( _ )\ (x86_64 Ubuntu 9.10) Linux 2.6.32.3
^ ^ 17:24:01 up 4 days 2:37 1 user load average: 1.02 1.03 1.00
不借貸! 不詐騙! 不援交! 不打交! 不打劫! 不自殺! 請考慮綜援 (CSSA):
http://www.swd.gov.hk/tc/index/site_pubsvc/page_socsecu/sub_addressesa

tom knauf

2010-01-13 13:01:03 UTC

Hi,

If you do it in VB(.NET) do not forget to deploy the runtime libs of
VB(.net)

I would make a stansdalone exe in VB.net , deploy it to your customers
(Visual Studio will include all necessary dlls)
and just RUN or shell_execute() it from fox, you can let it read an
interfacefile created by foxpro with filenames,...

BTW , did you look here http://www.marshallsoft.com/ ?
They offer a client AND server component for TCP/IP including foxpro
examples.

HTH
Tom

Post by MikeA
True but I don't know how to wrap the VB to VFP
Mike

Post by MikeA
I'm able to check them myself and I have programmatically done that with
checksums but the problem is it slows the transfer rate way down and
becomes a heavy programming job for what should be just a few lines of
code. The vendor that supplies this sockets says I have to pass it a
Byte Array like a Visual Basic SAFEARRAY in variant (whatever that
means). Is there a way to do this in Visual FoxPro or perhaps an ActiveX
control to allow me to convert a string to this.
Thanks,
Mike

Post by MikeA
Does anyone have any ideas and/or solutions - I'm really getting kind of
desperate for a solution?

MikeA

2010-01-14 08:04:02 UTC

Marshall appears to work but the file transfer is very slow (not sure why).
I would prefer to find an ActiveX control that works. I'm just not sure why
this is so difficult.

All help is most appreciated.

Thanks,
Mike

Post by tom knauf
Hi,
If you do it in VB(.NET) do not forget to deploy the runtime libs of
VB(.net)
I would make a stansdalone exe in VB.net , deploy it to your customers
(Visual Studio will include all necessary dlls)
and just RUN or shell_execute() it from fox, you can let it read an
interfacefile created by foxpro with filenames,...
BTW , did you look here http://www.marshallsoft.com/ ?
They offer a client AND server component for TCP/IP including foxpro
examples.
HTH
Tom

Post by MikeA
True but I don't know how to wrap the VB to VFP
Mike

Post by MikeA
I'm able to check them myself and I have programmatically done that with
checksums but the problem is it slows the transfer rate way down and
becomes a heavy programming job for what should be just a few lines of
code. The vendor that supplies this sockets says I have to pass it a
Byte Array like a Visual Basic SAFEARRAY in variant (whatever that
means). Is there a way to do this in Visual FoxPro or perhaps an ActiveX
control to allow me to convert a string to this.
Thanks,
Mike

Post by MikeA
Does anyone have any ideas and/or solutions - I'm really getting
kind
of
desperate for a solution?

Man-wai Chang to The Door (24000bps)

2010-01-14 08:33:28 UTC

Post by MikeA
Marshall appears to work but the file transfer is very slow (not sure why).
I would prefer to find an ActiveX control that works. I'm just not sure why
this is so difficult.

Use Visual C++ to write your own...

--
@~@ Might, Courage, Vision, SINCERITY.
/ v \ Simplicity is Beauty! May the Force and Farce be with you!
/( _ )\ (x86_64 Ubuntu 9.10) Linux 2.6.32.3
^ ^ 16:33:01 up 5 days 1:46 1 user load average: 1.00 1.01 1.00
不借貸! 不詐騙! 不援交! 不打交! 不打劫! 不自殺! 請考慮綜援 (CSSA):
http://www.swd.gov.hk/tc/index/site_pubsvc/page_socsecu/sub_addressesa

MikeA

2010-01-14 17:09:07 UTC

I prefer to do it all in VFP and I should be able to with the right controls
if I can find them.

Mike

Post by MikeA
Marshall appears to work but the file transfer is very slow (not sure why).
I would prefer to find an ActiveX control that works. I'm just not sure why
this is so difficult.

Use Visual C++ to write your own...
--
@~@ Might, Courage, Vision, SINCERITY.
/ v \ Simplicity is Beauty! May the Force and Farce be with you!
/( _ )\ (x86_64 Ubuntu 9.10) Linux 2.6.32.3
^ ^ 16:33:01 up 5 days 1:46 1 user load average: 1.00 1.01 1.00
http://www.swd.gov.hk/tc/index/site_pubsvc/page_socsecu/sub_addressesa

Patrick Heinrichs

2010-09-23 15:11:51 UTC

Hi Mike,

I read you post with interest as I will be embarking on a similar project with VFP and am curious whether you found a solution to the corrupted data problem you were experiencing.

Patrick

Post by MikeA
Does anyone have sample code using winsock to transfer a file with a VFP
program clientsample.prg and serversample.prg? I am still having problems
with no answers from prior posts and every time I think I am that much closer
to a solution it just does not happen. I would think someone would have some
sample code somewhere. I searched the UT last night but found nothing and
I am just getting a little tired of staying up til well after 3AM every
night.
All help would be greatly appreciated.
Thanks,
Mike

Post by Richard Stecenko
Hi Mike,
Maybe you already know about this: there is a VFP sample at Microsoft.
http://support.microsoft.com/kb/315124
Richard Stecenko
Interactive Computer Services Inc.
Winnipeg, Canada
204.453.2052

Actually I was not aware of that example. it is a great "starter." The
problem is that example only transfers a string of data. The problem I am
having is when I transfer a large file like 75 megs or so and some of the
"chunks" of data are not being properly sent.
I am stumped.
Thanks,
Mike

I should have mentioned -that is why I am looking for a sample that transfers
an entire file and not just a string of data.
Thanks,
Mike

Post by Richard Stecenko
Just for testing purposes, could you do a filetostr('MyFile') and then
send it using the Microsoft sample?
Richard Stecenko
Interactive Computer Services Inc.
Winnipeg, Canada
204.453.2052

Post by JÃ¼rgen Wondzinski
Hi Richard
But that would limit him to 16Mb (the max length of a string in VFP), but he
wants to transfer files with 75Mb.
--
wOOdy
Visual FoxPro Evangelist
Microsoft "Most Valuable Professional" 1996 to 2009
"*??)
?.???.?*??) ?.?*?)
(?.??. (?.?` *
..?`.Visual FoxPro: it is magic !
(?.?``??*

Post by MikeA
Not just that but I could easily go beyond the 16mb by using the low level
VFP file i/o commands which I do in my application but socket code has to be
rewritten such that it checks for ready to send events, etc. I do all that
with winsock in my test program and it does not work. There are chunks of
data that are not being properly sent. A lot of times (but not always) the
chunks are being sent out as zeros in the buffer even though they are read
as non-zero bytes.
it is just perplexing.
Mike

Post by Richard Stecenko
From the samples that I have seen, there is no send file method.
Instead, the client reads the file and sends it in packets. From the
samples that I have seen, the client sends a packet, the server echoes
back, and the client either re-sends or sends another packet.
In the Microsoft sample, all that is missing. it is primitive.
But if you look at the Microsoft sample, in the DataArival event (or
strData = SPACE(256) && Define string to pass to GetData
ThisForm.Edit1.Value = strData
Now, and I do not know how or why, if the getData() fails (screws up,
gets confused), then ThisForm.Edit1.Value is going to be a bunch of
spaces.
"A lot of times (but not always) the chunks are being sent out as
zeros in the buffer even though they are read as non-zero bytes."
So, it may be that the client reads the non-zero bytes and sends the
non-zero bytes, but the server fails (screws up, gets confused) and is
left with whatever strData was before the GetData.
So, it seems to me that it would be interesting to know what Mike has
in his code before the GetData in the string passed to GetData.
If for example, it was
strData = replicate('A', 256) && Define string to pass to GetData
ThisForm.Edit1.Value = strData
would he find As in the chunks?
If it is As then the problem is with the server, not with the client.
And probably some more sophisticated semaphoring is required.
Richard Stecenko
Interactive Computer Services Inc.
Winnipeg, Canada
204.453.2052

Post by Richard Stecenko
http://fox.wikis.com/wc.dll?Wiki~WinSockOCXSample
Richard Stecenko
Interactive Computer Services Inc.
Winnipeg, Canada
204.453.2052

Post by Gene Wirchenko
How is the data being sent, by TCP or UDP? The latter is not
guaranteed to be delivered.
Sincerely,
Gene Wirchenko

Post by MikeA
Here are my comments and I appreciate your continued advice and suggestions.
First, the Microsoft example has bugs in it for sending a file (or any
string of decent length). I have tested this even on a small file as little
as 7K. One must use the CreateBinary function when sending data or the data
will not match. This is not in the Microsoft example but is easy enough to
fix.
I have tried to modify the code so as to add some simple code to do a low
level file i/o and currently the file lengths are not matching in the
example with my modified code (when sending a 4 meg file). I am still
working with this.
I understand that and that is why I am looking for a simple program that
shows how to send a file. When I create my own and ttry, it is not working.
These samples then are bad. There is no reason for this to be done at the
application layer because TCP/IP automatically verifies the packets and has
it is own checksums. I have written a program like that with pretty good
success (about 99% success) where I do echo a checksum and resend if
necessary. The problem with that is first it is already done at the TCP/IP
layer and second it is going to slow the transfer rate way way down (that is
the biggest problem) along with adding unnecessary overhead and code to the
program.
So my search continues but I appreciate your feedback and I will check out the
Wiki samples.
Thank you again,
Mike

I am supplying an IP address and port so would that mean TCP/IP? I have put
a sniffer on my network to check this out and I see WRONG DATA is being sent
out from the sender. I see packets of zeros are being sent every so often
so the problem is happening from the client side sometime before the socket
sends its data out it has the wrong data but I am not sure why.
Thanks,
Mike

I looked a the Wiki link you sent - I have played around with EEVA and gotten
it to work nicely but that is a webserver app and it works great with VFP
for web apps. However, that has nothing to do with sending a large chunk of
data such as a zip file from a client app to a server app.
Still searching for a solution - all help is greatly appreciated.
Thanks,
Mike

I have been playing around with this for an hour.
I am pretty sure that I am sending the file in chunks.
But how do I get the server to stop receiving long enough to do
something with the chunks?
I keep getting an error of string too long. The server is just sitting
there on the getData until it chokes (at around 48 megabytes).
On the client side, do you send and then do a getData for a signal
from the server that it is ok to send the next chunk?
Richard Stecenko
Interactive Computer Services Inc.
Winnipeg, Canada
204.453.2052

Post by Gene Wirchenko
No.
IP Address, Port Number, and whether TCP or UDP are what are
needed. The calls that you use might establish a TCP or a UDP socket.
Which is it?
If the latter, you will have problems when the network gets
congested. You could have it at any time, but a non-malicious
implementation is not going to drop UDP data for no good reason. UDP
does not have guaranteed delivery, so it can be dropped, and there
will be no retry. TCP has retry.
How do you know that the zeroes are connected with your big file?
If you can be sure, then you have a problem up-line of
transmission.
[snip]
Bottom-posting makes it easier to follow technical discussions.
Please use it.
Sincerely,
Gene Wirchenko

You do not have get the server to stop receiving; the socket waits for the
server to accept the input in the buffer. Just save the data using either
strtofile or low level I/O. You can use either since strtofile will append
to a file but low level I/O is faster in the long run but for test purposes
it will not matter.
On the client side you just send and send - you do not wait for the server to
say it is okay to send the next chunk. You just send it and the client side
does have an error event that is fired if the socket on the sending side
says the buffer is full and it needs to wait. I played with this with
winsock and even with 75 megs when I have both client and server running on
my machine it never fires an error to wait but the mabry socket does fire an
error to wait due to buffer full. However, it is all irrelevant because when
the data arrives at the server and I write out the data it is corrupted
which is all part of my problem.
Thanks for the continued help and support as I hope to find a solution.
Mike

I know for a fact the protocol with the Winsock is tcp/ip and I am almost
positive it is TCP/IP with the mabry socket. However, it would not matter
because the problem is not guarantee of delivery. The problem is at the
sending side it is sending out every so often chunks of zeros and sometimes
other bad data - my sniffer detected this.
Mike

Anyone have any ideas as to how to transfer a file through winsock? I do not
know where else to look or what else to do to troubleshoot this problem.
Thanks,
Mike

Post by Man-wai Chang to The Door (24000bps)
I would use the built-in command-line ftp client....
--
@~@ Might, Courage, Vision, SINCERITY.
/ v \ Simplicity is Beauty! May the Force and Farce be with you!
/( _ )\ (x86_64 Ubuntu 9.10) Linux 2.6.32.2
^ ^ 17:49:01 up 13 days 1:03 2 users load average: 1.09 1.04 1.01
http://www.swd.gov.hk/tc/index/site_pubsvc/page_socsecu/sub_addressesa

Post by Man-wai Chang to The Door (24000bps)
Try google "ftp using winsock" as well...
--
@~@ Might, Courage, Vision, SINCERITY.
/ v \ Simplicity is Beauty! May the Force and Farce be with you!
/( _ )\ (x86_64 Ubuntu 9.10) Linux 2.6.32.2
^ ^ 21:59:01 up 13 days 5:13 2 users load average: 1.14 1.05 1.01
http://www.swd.gov.hk/tc/index/site_pubsvc/page_socsecu/sub_addressesa

Post by Jeff Grippe
This is a more complex problem that fits a newsgroup posting but I can
explain some concepts and provide some samples.
I have message types defined and a packet protocol that I use which may be
too "heavy" for sending large files. I break up what I am sending into
packets of a fixed length. I send one packet and wait for a confirmation
before sending the next packet. Each instance of my application has an IP
address and port which it listens on. All applications can both send and
receive.
You need to bind the winsock control to some port. I have multiple users
each of whom binds to a separate port. I begin with port 1002 and look for
an available port. Once a user has bound to a port, I save the results in a
table. I use UDP Protocol and I save all of my "messages" in a table. This
is not necessary but I like having the trail. What I call a message is just a
string of characters. There is no reason why it could not be the contents of a
file.
*** PORT BINDING
this.nIPPort = 1002 && first port to try binding to
this.oMSWinSock.protocol = 1 && UDP
this.oMSWinSock.Bind(this.nIPPort)
*** ERROR HANDLING IN CASE THE PORT DOESN'T BIND
LPARAMETERS nError, cMethod, nLine
DO CASE
CASE nError = 1429 && Problem binding port
this.nIPPort = this.nIPPort + 1 && try next sequential port
RETRY
OTHERWISE
=MESSAGEBOX(STR(nError,4,0)+" "+cMethod+" "+STR(nLine,4,0)+MESSAGE())
ENDCASE
*** SENDING DATA
this.oMSWinSock.RemoteHost = lcToIP && whatever ip address you are
sending to
this.oMSWinSock.RemotePort = lnToPort && whatever port you are sending to
*** My particular message protocol
lcFullMessage = "<MESSAGE>"+;
"<"+ALLTRIM(STR(lnPacket,5,0))+">"+;
"<"+ALLTRIM(STR(lnPacketCount,5,0))+">"+;
"<"+this.oMSWinSock.localip+">"+;
"<"+ALLTRIM(STR(this.nIPPort,6,0))+">"+;
"<"+pcmessageid+">"+;
"<"+SYS(2007,lcMessageSegment)+">"+;
"|"+lcMessageSegment
*** Remember that I save my messages
SELECT SentMessages
APPEND BLANK
REPLACE type WITH "MESSAGE", ;
packet WITH lnPacket, ;
packetcount WITH lnPacketCount, ;
ToIP WITH this.oMSWinSock.RemoteHost, ;
ToPort WITH this.oMSWinSock.RemotePort, ;
MessageID WITH pcMessageID, ;
CheckSum WITH SYS(2007, lcMessageSegment), ;
MessageData WITH lcMessageSegment, ;
RawData WITH lcFullMessage
*** SENDING THE MESSAGE
this.oMSWinSock.SendData(lcFullMessage)
The message comes in to the WinSock control and fires the DataArrivalEvent.
This in turn uses the GetData method to return the data. That is where your
code has to go to pickup the data that was sent.
*** RECEIVING THE DATA IN THE DataArrivalEvent
*** ActiveX Control Event ***
LPARAMETERS bytestotal
lcData = SPACE(256) && Define string to pass to GetData
* get fields and data
lcType = STREXTRACT(lcData, "<", ">", 1)
*** Processing the data using my protocol
DO CASE
CASE lcType = "MESSAGE" && message
lnSplitPos1 = AT("|", lcData, 1)
lnPacketNumber = VAL(STREXTRACT(lcData, "<", ">", 2))
lnTotalPackets = VAL(STREXTRACT(lcData, "<", ">", 3))
lcIPFrom = STREXTRACT(lcData, "<", ">", 4)
lnPortFrom = VAL(STREXTRACT(lcData, "<", ">", 5))
lcMessageID = STREXTRACT(lcData, "<", ">", 6)
lcCheckSum = STREXTRACT(lcData, "<", ">", 7)
lcMessage = SUBSTR(lcData, lnSplitPos1+1, LEN(lcData) - lnSplitPos1+1)
*** only necessary if you want to save the data like I do. Very useful
for debugging
SELECT RecvMessages
APPEND BLANK
replace type WITH lcType, ;
packet WITH lnPacketNumber, ;
packetcount WITH lnTotalPackets, ;
FromIP WITH lcIPFrom, ;
FromPort WITH lnPortFrom, ;
MessageID WITH lcMessageID, ;
CheckSum WITH lcCheckSum, ;
MessageData WITH lcMessage, ;
RawData WITH lcData

Post by MikeA
Jeff - thank you for your post.
My code works fine when I do not send too much information. It also works if
I do send information and use my own ACK and NAK. But, I should not have to
do that. My only conclusion at this time is Winsock has bugs in the control
itself causing problems when sending a lot of data.
But, I am still researching all this.
Thanks,
Mike

Post by Jeff Grippe
I had similar problems which is what led me to develop my "message" wrapper.
I calculate the number of packets that I will be sending and include a packet
number and count with each packet. The idea was that if I later found that I
was missing one, I could go back and request the missing packet.
As you found out, when you make the packets small enough, the winsock
control seems to get the job done accurately.
My senders were built so that they could be sending and receiving from
multiple other instances of the program at one time. A sender could initiate
a 40 packet send to Client A and a 50 packet send to Client B. As each
client sends back confirmation that they received the current packet, then
the sender sends the next packet.
I have never had to code the NAK. By using 100 byte messages, I have gotten
consistantly correct data. I think my message structure is too heavy for
sending large files, however. For a 100 byte message, I am probably sending
between 30 and 50 bytes of wrapper.
Jeff

Post by MikeA
When you are talking about a huge file transfer like 75 megs or several
hundred megs no matter how small the packets are it just does not work. I
think winsock is buggy and the only way to get it to work is to either use
nak/ack with large files. But, I am just not understanding why. There is no
reason the socket should be buggy regardless of the size of the file because
all the validations are done at the TCP/IP layer so there is no reason we
should have to do any checksums at all.
Mike

Does anyone have any ideas and/or solutions - I am really getting kind of
desperate for a solution?
Thanks,
Mike

I have been playing around with Winsock since you made the first post
and I think that you are right when you suggest that Winsock will not
work for large files.
I could get about 20 to 30 megs of data going from the two forms
(client and server) in the Microsoft VFP sample. But as soon as I
started sending more, it just did not work reliably.
I never could get the sendComplete to fire consistently. I could not
find any samples in VFP that did more than send very small chunks of
data. The VB samples I saw were also simple (i.e. trivial amounts of
data) or involved all sorts of extra acking and naking, which, you
suggest should not be necessary). I did find one web site where the
developer claimed that Winsock will only work reliably in C. I read
more stuff on the Nagle Algorithm than I ever thought possible.
I'd say, give it up Mark. Find some other way to move those files.
Richard Stecenko
Interactive Computer Services Inc.
Winnipeg, Canada
204.453.2052

No - I cannot give this up. First, I HAVE gotten it to work but only at half
speed by doing my own checksums.
I know that I can use some "heavy programming" to speed it up a bit but
TCP/IP should do all the checksums for me. At Catalyst.com (the control I am
testing now and at least there is support unlike mabry which is currently
out) they claim it works fine for 100 meg files or more.
So - now I"m thinking that maybe the problem is from where the string gets
passed to the socket. Maybe I need to try another way to load the socket's
buffer. Perhaps ComArray or just a pointer to a spot in memory but I posted
that in another thread. Believe me, I have been banging my head against the
wall with this one and I can get it to work by doing my own checksums but
it is not as efficient speed wise or programming that should not be
necessary.
So I am going to continue my search. But, I am thinking maybe if I can pass
the string a different way then it might work. I cannot use FTP or HTTP
sending or just a copy through a VPN because many of my clients may not have
any of that set up (in fact, most do not even have their own FTP and it
does not make sense to send it to a public FTP because then again it is going
to take twice as long and I can do that now. The simplest way is a socket.
As you can see - I do not give up easily (and never have) but I certainly
appreciate your support in this and I too disabled the Nagle and tried all
kinds of stuff. The only full success I had was when I created my own
checksums.
Please see my most recent thread.
Thank you again.
Mike

Post by MikeA
Richard - I almost forgot - in addition to my prior post - I know that
winsock is not reliable but I am using third party sockets and they too are
not working. I know for a fact that mabry is way more stable than winsock
but again, it too has problems with file transfer.
Mike

Post by Richard Stecenko
OK, so if Winsock is not reliable, and your third party sockets are
not reliable, what is left over?
you can have a fast transfer or a reliable transfer. You just cannot
have both. Again, from what I have read, it is possible to just jam in
too much data. That's why that one web site claimed that programs that
use Winsock should be written in C.
Now, do not take what I am saying at face value or as gospel. I was
curious about what you are trying to do and read a few things and
tried a few things. But, (I hate to say this), your problem is not my
problem. And if I were you, I'd try another approach.
Richard Stecenko
Interactive Computer Services Inc.
Winnipeg, Canada
204.453.2052

Post by MikeA
Thank you again for your continued support. If you have a better approach
to this then I am all ears. But, here are my thoughts. First, whether Nagle
is enabled or not, File Transfer through the socket still does not work
(without implementing your own checksums). But, like I say, I can get it to
work and have by using my own checksums. So, the socket solution will work
it is just I want the code as "clean" as possible without sacrificing speed.
A socket is the most simple "out of the box" solution. For example,
programs like pc Anywhere or gotomypc, etc. do not force the user to have
their own FTP or HTTP when you want to utilize them to do a file transfer
from or to the host. I do not want my customers to have to be dependent on
some additional setup such as FTP servers because this is a major piece I am
adding to my Foxpro database application.
As far as what I am trying to do - it goes like this. My clients right now
have the ability to copy my entire database application to their laptop
computers then work "offline" without any internet or network. Then they
can take their laptop computer back to their site and sync the data. I want
to allow them to sync from anywhere via the internet in as few steps as
possible. The ability for them to do this is huge. Also, a socket is ideal
for this. We are talking about several steps internally here so for
example, client may need to transfer a file to the server, server then
processes that file then server sends client a few files and client
processes those and so on back and forth in a very ordered fashion until the
sync is done. The files must be processed in a specific order and the
client server approach is idea. But again, I am already doing it with
checksums it is just the programming gets very messy and twice as bad if I
try and optimize the speed programmatically and before I dedicate a lot of
time to trying to optimize the code I would like to find a way to do it
RIGHT where TCP/IP does all the checksums for me so as to have the speed
optimized.
I agree about it being my problem and this forum is great as we are all here
to help each other but unless one comes up with a better solution to what
I am trying to do - I am going to do my best to optimize the socket as a
solution.
On another note, honestly, it is hard for me to believe that for all the time
VFP has been around, something as basic as a "file transfer" using sockets
with VFP has not been posted before. I know that several threads have been
written on how to do messaging in Visual FoxPro and again sockets are an
ideal solution and work great. I am just trying to extend that a little
further to a file transfer.
Thanks again,
Mike

Post by Man-wai Chang to The Door (24000bps)
What if you check the errors yourself rather than relying on the stack?
--
@~@ Might, Courage, Vision, SINCERITY.
/ v \ Simplicity is Beauty! May the Force and Farce be with you!
/( _ )\ (x86_64 Ubuntu 9.10) Linux 2.6.32.2
^ ^ 18:23:01 up 18 days 1:37 2 users load average: 1.03 1.01 1.00
http://www.swd.gov.hk/tc/index/site_pubsvc/page_socsecu/sub_addressesa

I am able to check them myself and I have programmatically done that with
checksums but the problem is it slows the transfer rate way down and becomes
a heavy programming job for what should be just a few lines of code. The
vendor that supplies this sockets says I have to pass it a Byte Array like a
Visual Basic SAFEARRAY in variant (whatever that means). Is there a way to
do this in Visual FoxPro or perhaps an ActiveX control to allow me to
convert a string to this.
Thanks,
Mike

Post by Man-wai Chang to The Door (24000bps)
If it worked, then use it until there was a better solution. :)
I thought you were using the standard M$ WinSock ActiveX control. Seems
that you were not....
--
@~@ Might, Courage, Vision, SINCERITY.
/ v \ Simplicity is Beauty! May the Force and Farce be with you!
/( _ )\ (x86_64 Ubuntu 9.10) Linux 2.6.32.3
^ ^ 16:18:01 up 1:31 0 users load average: 1.00 1.00 1.00
http://www.swd.gov.hk/tc/index/site_pubsvc/page_socsecu/sub_addressesa

A simple checksum should not slow the transfer down that much. I designed a
more verbose messaging package because I knew my messages would be short and
I wanted a richer package.
If you break the file into packets and use a lighter weight protocol in
which you have 100 bytes (or a little more) of message and only a few bytes
of checksum, then you should get decent performance. You are not going to see
anywhere near the performance that you would get from something like a
mapped drive no matter what you do. But it should work well enough with your
basic ACK/NAK that you have setup.
FWIW, I have run into the same limitations that you have and I do not have a
better solution.
Jeff

Post by MikeA
The problem I think boils down to strategy and programming algorithms. If I
cannot just flood the socket with data and let TCP/IP do the work for me of
verifying the data then it seems like I have to have another strategy for
continuously keeping data flowing along the socket otherwise it will slow to
Send Packet A to server
Have server verify checksum
Send checksum back to client
Have client resend if checksum is bad
That would be terrible because the server would not be continuing to receive
data. it is really lousy that I cannot just let TCP/IP verify all this for
me.
What other strategies have you used to make file transfer work for you?
it is just hard to imagine having to resend thousands of packets due to
checksum errors.
Thanks,
Mike

Post by MikeA
If I use the strategy of flooding the socket and tracking checksums at the
server then it is a lot of heavy programming that it seems like I should not
have to do since TCP/IP already does this for me.
Server receives 1000 packets and say 50 are bad
Server request to resend those 50 packets from the client
Server now needs to rebuild the file to merge those 50 packets where they
belong in the file (this requires a re-read of the file from the client and
rewrite of the file at the server)
Server needs to repeat the steps if of the 50 packets some of those are bad
and continue repeating until all the checksums are good.
This just seems like a lot of heavy programming that should not be required
if TCP/IP is already doing the checksums.
Mike

Post by Jeff Grippe
Have you considered using FTP which was designed for file transfer?
There are several good libraries for FTP such as WW Internet tools from West
Wind. I am sure there are free ones too. That should give you the performance
you want and the stability that you need.
Jeff

Post by MikeA
I have considered it but the problem is a lot of my users do not have an FTP
server.
On another note - the vendor that provides the socket I am testing says that
it works fine in VB with the SafeArray. So, now I am thinking maybe there is
an ActiveX control I can use to populate the socket's buffer.
Otherwise, I will just have to use the sockets and do my own checksums. it is
really a waste because I would love to just do my own checksums and then
just turn of the TCP/IP checksums otherwise it is just a lot of unnecessary
data slowing down the transfer.
Thanks,
Mike

Post by MikeA
Jeff - I have a question - why break the file into 100 byte packtes to send
instead of say 1000 bytes with a checksum? Then I am sending one checksum
rather than 10. Any thoughts on this? Also, I am trying to experiment with
the idea of possibly getting a false positive with a checksum but I"m not
sure how likely that is but then again the more packets that are sent the
more likely that I could get a false positive. That alone would seem to
mean the odds of a false positive are 10 times greater using 100 byte
packets rather than a 1000 byte packet.
Thanks again for all insights.
Mike

Post by Richard Stecenko
Hi Mike,
Why do not you write it in VB using SafeArray to see if it really works. If
it does, create a wrapper for it in FoxPro. On the flip side, it will run
out of process.
If you are writing this kind of code, VB should be a piece of cake even if
you do not know VB. Plus, you probably have VB sample code from the vendor.
Good Luck,
ED.
I have considered it but the problem is a lot of my users do not have an FTP
server.
On another note - the vendor that provides the socket I am testing says that
it works fine in VB with the SafeArray. So, now I am thinking maybe there is
an ActiveX control I can use to populate the socket's buffer.
Otherwise, I will just have to use the sockets and do my own checksums. it is
really a waste because I would love to just do my own checksums and then
just turn of the TCP/IP checksums otherwise it is just a lot of unnecessary
data slowing down the transfer.
Thanks,
Mike

True but I do not know how to wrap the VB to VFP
Mike

VB could produce a standard DLL. Just make sure that the DLL does not
have user interface.
--
@~@ Might, Courage, Vision, SINCERITY.
/ v \ Simplicity is Beauty! May the Force and Farce be with you!
/( _ )\ (x86_64 Ubuntu 9.10) Linux 2.6.32.3
^ ^ 17:24:01 up 4 days 2:37 1 user load average: 1.02 1.03 1.00
http://www.swd.gov.hk/tc/index/site_pubsvc/page_socsecu/sub_addressesa

Post by Jeff Grippe
I, like you, found that WinSock was not reliable for large amounts of data.
My desicion to use 100 byte packages was somewhat random although I wanted
the ability to compare what the sender and receiver were getting and going
much over 100 bytes would have made visual inspection difficult.
There is no reason not to use 1000 byte packets once you have convinced
yourself that you will get reliable performance with that size.
Since my messages are stored in a database, I am now sending very small
messages which only serve to let the receiver know that a database record
has been updated or created. I used to send the contents of the database
record (rtf data) until I realized that I could just as easily put the data
into a memo field and simply send a message telling the receiver where to
find the data.
My messages are now all well under 100 bytes so I had no need to go larger.
I do not know that you need a dedicated FTP server to use FTP for file
transfer. I think that various tools that are available let you do both
sending and receiving via FTP. I am not sure, however, as I have only uploaded
to a server and pulled down from a server using FTP.
Jeff

I do not know if it is possible to do FTP from just "computer to computer"
without an FTP server. Also, I realized thinking further about the solution
to creating my own checksums can still cause problems.
Suppose CLIENT sends 10,000 packets of 1000 bytes to SERVER (or 10 megs).
Suppose SERVER says 100 packets are bad. SERVER now needs to send CLIENT
the packet numbers that are bad so that CLIENT can resend them. So, that
would be 100 integers to identify the packet numbers that need to be resent.
Each integer may need to be identified by four bytes or 4 * 100 = 400 bytes
that need to be sent to the CLIENT to specify which packets need to be
resent. Now, the SERVER could in theory send 400 bytes to CLIENT where
some of the bytes are corrupted. In other words, the "resend information"
could be corrupted.
What a mess....uggg.
Mike

Jeff - I am finding the third party controls (at least two) also are not
reliable for large amounts of data. Is there a third party control that IS
reliable that I can use?
Thanks,
Mike

Post by MikeA
Marshall appears to work but the file transfer is very slow (not sure why).
I would prefer to find an ActiveX control that works. I am just not sure why
this is so difficult.
All help is most appreciated.
Thanks,
Mike

Post by Man-wai Chang to The Door (24000bps)
Use Visual C++ to write your own...
--
@~@ Might, Courage, Vision, SINCERITY.
/ v \ Simplicity is Beauty! May the Force and Farce be with you!
/( _ )\ (x86_64 Ubuntu 9.10) Linux 2.6.32.3
^ ^ 16:33:01 up 5 days 1:46 1 user load average: 1.00 1.01 1.00
http://www.swd.gov.hk/tc/index/site_pubsvc/page_socsecu/sub_addressesa

Post by ED
There are many ways to do it, but before crossing that bridge, I suggest you
first write a simple VB program to check if the control really works. Just a
few lines to send that big file of yours.
ED.
True but I do not know how to wrap the VB to VFP
Mike

Post by MikeA
I prefer to do it all in VFP and I should be able to with the right controls
if I can find them.
Mike
Submitted via EggHeadCafe - Software Developer Portal of Choice
Silverlight Binary Serialization and Compression with WCF Services
http://www.eggheadcafe.com/tutorials/aspnet/96487d4c-d92f-4ca5-85b7-0fef6f42d6c3/silverlight-binary-serialization-and-compression-with-wcf-services.aspx

2010-01-14 08:39:50 UTC

There are many ways to do it, but before crossing that bridge, I suggest you
first write a simple VB program to check if the control really works. Just a
few lines to send that big file of yours.
ED.

"MikeA" <***@appellsoftware.com> wrote in message news:***@TK2MSFTNGP02.phx.gbl...
True but I don't know how to wrap the VB to VFP

Mike

Post by MikeA
I'm able to check them myself and I have programmatically done that with
checksums but the problem is it slows the transfer rate way down and
becomes a heavy programming job for what should be just a few lines of
code. The vendor that supplies this sockets says I have to pass it a
Byte Array like a Visual Basic SAFEARRAY in variant (whatever that
means). Is there a way to do this in Visual FoxPro or perhaps an ActiveX
control to allow me to convert a string to this.
Thanks,
Mike