Hi, I've created a set of scripts for configuring and updating dynamic DNS in Kamikaze. A UCI config file (/etc/config/ddns) can be used to configure what service to connect to (e.g. dyndns.org, no-ip.com etc), how often to check for updates, and how often to force an update.  The script uses wget to perform updates.  Specifically, I've created a new wget package that can connect via ssl, but doesn't require openssl which takes up a megabyte of disk space.  Instead, the new package links to matrixssl, which is only 90K.  I've also created a new version of libmatrixssl, as the old version is very out of date.  I've also verified that the two existing packages that link to matrixssl (matrixtunnel and mini_httpd) work fine with the new version.  I have posted a detailed description of the new packages here. Both source code and ipkg files for the three packages are available.  You can download the source code here, and you can download the (binary) pacakge files here.

Nice I may give it a try later.

Just a note. ip-up scripts are not used with Kamikaze. Use hotplug scripts instead.

Instead of wan_proto=$(uci get network.wan.proto) in files/etc/hotplug.d/iface/25-ddns you should use UCI API functions.

So instead of:

#!/bin/sh

. /usr/lib/ddns/dynamic_dns_functions.sh

wan_proto=$(uci get network.wan.proto)

if [ "$INTERFACE" = "wan" ] && [ "$ACTION" = "ifup" ] && [ "$wan_proto" != 'pppoe' ]
then
    start_daemon_for_all_ddns_sections
fi

use something like this:

#!/bin/sh

. /usr/lib/ddns/dynamic_dns_functions.sh
. /etc/functions.sh

config_load "network"
local section="wan"
config_get "wan_proto" "$section" "proto"

if [ "$INTERFACE" = "wan" ] && [ "$ACTION" = "ifup" ] && [ "$wan_proto" != 'pppoe' ]
then
    start_daemon_for_all_ddns_sections
fi

(Last edited by forum2008 on 28 Jan 2008, 09:05)

wget-matrixssl

I see that you modified wget to use matrixssl. Is it possible that you create a patch for this instead of copying the files/* over wget?

Ok. Already created the wget diff by myself. Next I will fix the net/wget/Makefile to build three versions of wget. wget, wget-matrixssl and wget-openssl.

Once finished I will commit it to subversion if it's ok for you. Its very interesting to have the choice to link wget against matrixssl.

Hmm, ok. I updated matrixssl to 1.8.3 and applied the wget patch. However, on brcm47xx target wget starts but it's doing nothing.

Here is the patch I applied to the trunk build-system: http://openwrt.pastebin.com/f324a189e

strace output: http://openwrt.pastebin.com/f67db9499

(Last edited by forum2008 on 28 Jan 2008, 12:18)

Hmm... I tested it (at least the ipkg builds that I link to) both on my development machine and on the brcm47xx build with a 2.4 kernel (on my ancient WRT54G v2).  If you just install the ipk files do you get the same errror?  Also, did you put quotes around whatever url you were trying to retrieve with wget?  I've noticed sometimes wget (standard and matrix-ssl version) hangs if you don't quote the url.

I'll look more closely at the patches you apply above, and see if I can figure out what's going on.

I got errors when I tried applying the patch you provide above.  I created my own patch which you can download here.  It does the same thing you were trying to do (modifies the wget package with a patch, makes all 3 versions buildable).  To apply it you just download the latest version from the svn, copy the patch into the packages directory and run "patch -p0 < ddns-wget-matrix.patch" I checked out a fresh version of the svn, applied the patch and built the packages of interest. I tested those packages again on my system and the wget-matrixssl binary seems to work fine.  If you still have problems could you let me know as many details about your system as you can provide and what url you were trying to connect to?

Ok. I compiled a fresh checkout from trunk (revision 10297) for target brcm47xx [2.6] with your patch applied. Only minimal set of packages selected.

Wget-matrixssl still does not work. I tried 'wget --help' and it just hangs. I am out of ideas.

(Last edited by forum2008 on 28 Jan 2008, 23:36)

Both of my routers are running the 2.4 kernel, so the issue may have something to do with the 2.6 kernel.  I am going to install kamikaze 7.09 brcm47xx with a 2.6 kernel on one of my routers and see if I can replicate the error.

I tried installing the broadcom 2.6 kernel on one of my routers, and the packages still seem to work fine, so I'm at a loss to explain what's going on.  I'd really like to be able to replicate your problem so that I can fix it.  Could you try installing the binary ipkg files that I posted above?  I've tried those specific packages on your configuration (broadcom 2.6) and they worked for me.  Ideally you should be able to compile the packages, but if the binaries work then the problem is definitely related to how you are compiling them, and if they don't the problem is definitely elsewhere.  Also it would be very helpful if could you send me a tarball of your sdk/package directory, another of the binary packages you produce (so I can see if they hang on my systems) and a log of the output when you make clean and then run "make V=99" in your root SDK directory.  You can email me at <ericpaulbishop AT gmail.com>.  Thanks!

Is it possible that someone patches wget to link against CyaSSL (http://yassl.com/download.html)?

Do you want wget linked to CyaSSL, or would you be happy with a _working_ wget linked against matrixssl?

(I ask, because the problems that you experienced in January with ebishop's wget/matrixssl have recently been fixed - by ebishop! I now have a svn trunk wget compiled against the latest matrixssl library and it works perfectly).

J :-)

Guys, are those packages/patches going to be included in Kamikaze trunk as well? Thanks.

I'll find out... :-)

Hi.  I first linked wget to matrixssl as a way for ddns-scripts to update via https.  However, since there was no activity on this thread for a really long time I figured no one was too interested.  In the mean time I created a new ddns update client in C that links directly to matrixssl so I could eliminate the need for wget altogether. 

However, a couple of weeks ago jaime (above) emailed me, asking about it and thanks to him I figured out (finally) how to replicate the error and how to fix it.  The problem only occurs if you are using one of the trunk builds that uses uclibc v0.9.29 instead of v0.9.28 which kamikaze 7.09 uses.  Since I've been doing all my development on Kamikaze 7.09, finding the error drove me crazy.

Basically, the pthreads implementation in that version is a bit screwed up, matrixssl uses the mutex code in libpthread to ensure thread safety, and the two don't play nice together.  The way I got it working was by  creating a version of matrixssl that is not thread safe and linking wget to it.  Since wget doesn't use threads, this cannot cause any problems. 

The reason I wanted to  create a new ddns client was for my new web interface, Gargoyle, which I have just released.  (See below post, or visit http://www.gargoyle-router.com for more info)  Even though wget-matrixssl is not used by Gargoyle I have included the code for it on the Gargoyle svn, and the package is available for download from the Gargoyle package repositories (available for atheros 2.6 and broadcom 2.4 routers). 

If you are running on other hardware you will need to compile it for yourself.  You can checkout a copy of the gargoyle code (which  includes wget-matrixssl) from the svn by running the following command:
svn checkout https://svn.assembla.com/svn/gargoyle-router/trunk

All this, of course, skirts the question as to whether this  will be included in the upcoming release of Kamikaze.  The best answer I can give you right now is: I don't know.  I want to see what the overall community's reaction is to the Gargoyle code (love it/hate it/ignore it) before deciding whether to  post a bunch of patches  that add the code for it to the openwrt svn.  Granted, wget-matrixssl is a special case that's  not part of the main interface, but for now I'm going to wait and see.

If you REALLY want this included I suggest you make a patch yourself from the  gargoyle svn and send  it to the developers mailing list yourself.  I promise not to be offended, especially if  you mention where you got it in your email.  My guess is that the developers don't care too much who the code comes from so long as it works.  Be aware you'll need patches for both wget-matrixssl and libmatrixssl-nothread.

Hi,

I happened to be looking for anybody who had sewn matrixssl into either curl or wget,
to save me a bunch of work, and stumbled upon this forum.

I applied your patches to wget and got then got it to build under Windows.  It -almost- works
for my application.  But I've run into at least one bug in the sslpeek / sslread routines
that goof things up.

I'm wondering if you're interested in the details, any fix(es) I might come up with,
and/or if you have changed the code any since the initial burst of activity.
(I see it's listed as obsolete on your main website)

Thanks, hope all is well,
Bill

Hi.   I would be interested to know what errors you are getting, and what context you are using wget in that generates the errors.  Even though I am not actively developing my wget linked to matrixssl, I would like to do what I can to make sure it works properly.  I am curious if what you are seeing are windows-specific bugs or not (I haven't tested this code on anything but linux).   Also, make sure you are patching the right version of wget -- things may have changed over time since I made that patch last summer, and you may have better luck with the older version the patch was meant for.

So, I'll help if I can, but my time is very limited right now.  I can't promise any help, especially if the bugs are windows specific.

Thanks for your reply.
Yes, I did patch against wget-1.11.2.  I looked that over carefully before blinding applying the patches.

There were a few Windows related porting problems, but they were easily overcome.
One was straight C source files cannot declare new variables in a function after the first executable
statement.  MSVC6 sees that as a C++ feature.  So in two spots in matrixssl.c I moved some declarations
up to the top.

Next, in matrixssl_helper.c, _ssl_setSocketBlock() and _ssl_setSocketNonblock() make use of of calls
which Windows doesn't support.   I replaced:

    fcntl(fd, F_SETFL, fcntl(fd, F_GETFL) | O_NONBLOCK);

with:

    #ifdef WINDOWS
    unsigned long flags;
    flags = TRUE;
    ioctlsocket(fd, FIONBIO, &flags);
#else
    fcntl(fd, F_SETFL, fcntl(fd, F_GETFL) | O_NONBLOCK);
#endif // #ifdef WINDOWS

But most importantly, there seems to be a logic error in SSL_peek().
When wget is peeking to find the end of the HTTP headers, and if those headers are
longer than its initial 511 read request, the buffering logic in SSL_peek() causes
the data returned by the first call to peek to be lost on the next call(s).  It uses
read internally which advances the pointer, forgetting it's just a peek.

You have comments in one place:

    if(available > 0)
    {
        //we seem never to get here BUT the _ssl_read method in the helper functions
        //already had the provision to return with the data in the inbuf if it has
        //data in it.  This suggests  there might be data in it under some bizarre
        //condition and we certainly don't want to lose it if it exists.  So -- if there 
        //is data in there we'll just use that instead of trying to read more and 
        //putting it in the inbuf

I found the code just below which I believe to be the reason for that never being executed:

        if(read_bytes > 0)
        {
            memcpy(buf, newBuf, read_bytes);

            // WRL -- bug here
            // frees the wrong buffer and goofs up the pointers.
            // ssl->inbuf.start was advanced above inside SSL_read,
            // so now we have to back it up since we're only peeking.
            
            // free(ssl->inbuf.buf);
            free(newBuf);

            //ssl->inbuf.start = ssl->inbuf.buf = (unsigned char*)newBuf;
            //ssl->inbuf.end = (unsigned char*)newBuf+read_bytes;
            //ssl->inbuf.size = bufsize;
            ssl->inbuf.start -= read_bytes;
        }

This bug and these changes were found and tested with this URL:

    https://signin.ebay.com/ws/eBayISAPI.dll?SignIn

I'm not suggesting this is -the- fix, but I offer it for your consideration.

Thanks,
Bill

I've tested it, and the problem does indeed occur on linux as well as windows.  I've applied your fix, with a few adjustments to fix a memory leak (if nothing read newBuf doesn't get freed), and to simplify the first part of the SSL_peek function.  The new SSL_peek function (which works -- I tested it) looks like this:

int SSL_peek(SSL *ssl, char *buf, int bufsize)
{
    int read_bytes;
    unsigned long available = (unsigned long)(ssl->inbuf.end) - (unsigned long)(ssl->inbuf.start);
    if(available > 0)
    {
        read_bytes = available >= bufsize ? bufsize : (int)available;
        memcpy(buf, ssl->inbuf.start, read_bytes);
    }
    else
    {    
        char* newBuf = (char *)malloc(bufsize);
        read_bytes = SSL_read(ssl, newBuf, bufsize);
        if(read_bytes > 0)
        {
            memcpy(buf, newBuf, read_bytes);
            ssl->inbuf.start -= read_bytes;
        }
        free(newBuf);
    }
    return read_bytes;    
}

I've given you (Bill Lewis) full credit for finding & fixing this bug in the comment at the top of the source.  The new version has been committed to the gargoyle svn code repository (svn r114). Thanks for pointing out this bug and submitting this fix!

Eric,

Thanks for the credit and thanks for the official fix.
It's working very well.

Bill