Nabble - Lisp

Re: rawsock sniffer demo

2009-12-03T08:01:51Z

My advice to you and all others is to test all such things on a system
that you can easily reboot and that is not critical to anything before
making it part of any operational system or system used for any other
purpose. My implementations of this technology tend to be on systems
with bootable CDs and floppy-based configuration files (write locked
except during down-time). Thus I can push reset and try again on a
moment's notice. Improper use may disrupt not only the local machine,
but also surrounding networks, and who knows what else.

Also - in trying your test script, even after recompiling clisp with
rawsock built in, it failed...

>sudo clisp -K full /u/fc/lisp/sniffer.lsp inet 100
*** - UNIX error 43 (EPROTONOSUPPORT): Protocol not supported

>sudo clisp -K full /u/fc/lisp/sniffer.lsp packet 100
*** - RAWSOCK:SOCKET: Lisp value :PACKET is not found in table
"check_socket_domain":
((0 :UNSPEC) (1 :UNIX) (1 :LOCAL) (2 :INET) (3 :IMPLINK)
(4 :PUP) (5 :CHAOS) (9 :DATAKIT) (10 :CCITT) (23 :IPX) (6 :NS) (7 :ISO)
(7 :OSI) (8 :ECMA) (16 :APPLETALK) (30 :INET6) (12 :DECNET)
(13 :DLI) (14 :LAT) (15 :HYLINK) (17 :ROUTE) (11 :SNA) (33 :NETBIOS))

I tried various combinations including altering the values within the
program...

(defparameter *socket*
(let ((arg (pop *args*)))
(cond ((string= arg "inet")
(rawsock:socket :inet :dgram #x300))
((string= arg "packet")
(rawsock:socket :packet :raw #x300))
(t (error "invalid socket argument ~S" arg)))))
(unless (plusp *socket*)
(error "Invalid socket created for ~S: ~S" *args* *socket*))

Somehow, a demo should work across all such situations...

This seems to not error out on my system:
(defparameter *socket*
(let ((arg (pop *args*)))
(cond ((string= arg "inet")
(rawsock:socket :inet :dgram 0 #+ignore #x300))
((string= arg "packet")
(rawsock:socket :inet :raw 0 #+ignore #x300))
(t (error "invalid socket argument ~S" arg)))))
(unless (plusp *socket*)
(error "Invalid socket created for ~S: ~S" *args* *socket*))

FC
On Dec 3, 2009, at 7:38 AM, Sam Steingold wrote:

> Fred Cohen wrote:
>> (IN-PACKAGE "RAWSOCK")
>> (setf socket (rawsock:socket :inet :RAW 0 #+ignore :all)
>> device
>> (rawsock:make-sockaddr :UNSPEC
>> (make-array 14 :element-type '(unsigned-byte 8)))
>> buffer
>> (make-array 1518 :element-type '(unsigned-byte 8) :fill-pointer
>> 1518)))
>> (configdev socket "eth0" 1 1 ) ; eth0 no arp raw mode on socket
>> (defun getraw ()
>> (if (not (= -1 socket))
>> (setf (fill-pointer buffer) 1518
>> len #+ignore (rawsock::recvfr socket)
>> (rawsock:recvfrom socket buffer device)
>> (fill-pointer buffer) len)))
>> (defun print-buffer () (loop for i below len do (format t "~2,'0X
>> " (aref buffer i))))
>> ;; a rudimentary dump of all packets seen.
>> (loop while true do ((getraw)(print-buffer)))
>
> my sniffer.lisp does not call configdev.
> how does it affect its functioning?
> I do not want to try because your explanations did not allay my
> fears that the changes configdev introduces to the system are
> completely reversed by closing the socket.
>
>
>

- This communication is confidential to the parties it is intended to
serve -
Fred Cohen & Associates tel/fax: 925-454-0171
http://all.net/ 572 Leona Drive Livermore, CA 94550
Join http://tech.groups.yahoo.com/group/FCA-announce/join for our
mailing list

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: rawsock sniffer demo

2009-12-03T07:38:47Z

Fred Cohen wrote:

> (IN-PACKAGE "RAWSOCK")
> (setf socket (rawsock:socket :inet :RAW 0 #+ignore :all)
> device
> (rawsock:make-sockaddr :UNSPEC
> (make-array 14 :element-type '(unsigned-byte 8)))
> buffer
> (make-array 1518 :element-type '(unsigned-byte 8) :fill-pointer 1518)))
> (configdev socket "eth0" 1 1 ) ; eth0 no arp raw mode on socket
> (defun getraw ()
> (if (not (= -1 socket))
> (setf (fill-pointer buffer) 1518
> len #+ignore (rawsock::recvfr socket)
> (rawsock:recvfrom socket buffer device)
> (fill-pointer buffer) len)))
> (defun print-buffer () (loop for i below len do (format t "~2,'0X
> " (aref buffer i))))
> ;; a rudimentary dump of all packets seen.
> (loop while true do ((getraw)(print-buffer)))

my sniffer.lisp does not call configdev.
how does it affect its functioning?
I do not want to try because your explanations did not allay my fears that the
changes configdev introduces to the system are completely reversed by closing
the socket.

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

clisp-cvs Digest, Vol 44, Issue 2

2009-12-03T04:04:41Z

Send clisp-cvs mailing list submissions to
clisp-cvs@...

To subscribe or unsubscribe via the World Wide Web, visit
https://lists.sourceforge.net/lists/listinfo/clisp-cvs
or, via email, send a message with subject or body 'help' to
clisp-cvs-request@...

You can reach the person managing the list at
clisp-cvs-owner@...

When replying, please edit your Subject line so it is more specific
than "Re: Contents of clisp-cvs digest..."

CLISP CVS commits for today

Today's Topics:

1. clisp/modules/rawsock rawsock.c,1.126,1.127
rawsock.xml,1.60,1.61 (Sam Steingold)
2. clisp/modules/rawsock rawsock.c,1.127,1.128 (Sam Steingold)
3. clisp/src ChangeLog,1.7216,1.7217 (Sam Steingold)
4. clisp/modules/rawsock test.tst,1.58,1.59 (Sam Steingold)
5. clisp/modules/rawsock/demos - New directory (Sam Steingold)
6. clisp/modules/rawsock/demos README,NONE,1.1
sniffer.lisp,NONE,1.1 (Sam Steingold)
7. clisp/modules/rawsock Makefile.in,1.13,1.14 (Sam Steingold)
8. clisp/src ChangeLog,1.7217,1.7218 (Sam Steingold)
9. clisp/modules/rawsock rawsock.xml,1.61,1.62 (Sam Steingold)

----------------------------------------------------------------------

Message: 1
Date: Wed, 02 Dec 2009 21:06:04 +0000
From: Sam Steingold <sds@...>
Subject: clisp/modules/rawsock rawsock.c,1.126,1.127
rawsock.xml,1.60,1.61
To: clisp-cvs@...
Message-ID: <E1NFwOi-0003TA-15@...>

Update of /cvsroot/clisp/clisp/modules/rawsock
In directory sfp-cvsdas-2.v30.ch3.sourceforge.com:/tmp/cvs-serv13314/modules/rawsock

Modified Files:
rawsock.c rawsock.xml
Log Message:
checksum buffer is an ethernet frame starting with 14 byte MAC address

Index: rawsock.xml
===================================================================
RCS file: /cvsroot/clisp/clisp/modules/rawsock/rawsock.xml,v
retrieving revision 1.60
retrieving revision 1.61
diff -u -d -r1.60 -r1.61
--- rawsock.xml 1 Dec 2009 22:43:43 -0000 1.60
+++ rawsock.xml 2 Dec 2009 21:05:59 -0000 1.61
@@ -316,8 +316,9 @@
- &udp;</term>
<listitem><simpara>Compute the appropriate protocol checksum and record
it in the appropriate location. &buf-r; is assumed to be a suitable
- packet for the protocol, with the appropriate header etc.
- The typical packet you send is both &ip; and &tcp; and thus has two
+ ethernet frame (starting with a 14 byte MAC address) for the protocol,
+ with the appropriate header etc.</simpara>
+ <simpara>A typical packet you send is both &ip; and &tcp; and thus has two
checksums, so you would want to call <emphasis>two</emphasis> functions.
</simpara></listitem></varlistentry>
<varlistentry><term><code>(RAWSOCK:CONFIGDEV &sock-r;

Index: rawsock.c
===================================================================
RCS file: /cvsroot/clisp/clisp/modules/rawsock/rawsock.c,v
retrieving revision 1.126
retrieving revision 1.127
diff -u -d -r1.126 -r1.127
--- rawsock.c 1 Dec 2009 18:39:27 -0000 1.126
+++ rawsock.c 2 Dec 2009 21:05:58 -0000 1.127
@@ -1439,6 +1439,8 @@
#endif

/* ================== CHECKSUM from Fred Cohen ================== */
+/* these check sum functions operate on ethernet _frames_,
+ i.e., the 14 bytes MAC address followed by an IP datagram. */
static unsigned short ipcsum (unsigned char* buffer, size_t length) {
register long sum=0; /* assumes long == 32 bits */
unsigned short result;

------------------------------

Message: 2
Date: Wed, 02 Dec 2009 21:42:05 +0000
From: Sam Steingold <sds@...>
Subject: clisp/modules/rawsock rawsock.c,1.127,1.128
To: clisp-cvs@...
Message-ID: <E1NFwxZ-0005Tw-Du@...>

Update of /cvsroot/clisp/clisp/modules/rawsock
In directory sfp-cvsdas-2.v30.ch3.sourceforge.com:/tmp/cvs-serv21002/modules/rawsock

Modified Files:
rawsock.c
Log Message:
(get_socket_protocol) [HAVE_GETPROTOBYNAME]: fix argument order in the error message

Index: rawsock.c
===================================================================
RCS file: /cvsroot/clisp/clisp/modules/rawsock/rawsock.c,v
retrieving revision 1.127
retrieving revision 1.128
diff -u -d -r1.127 -r1.128
--- rawsock.c 2 Dec 2009 21:05:58 -0000 1.127
+++ rawsock.c 2 Dec 2009 21:42:02 -0000 1.128
@@ -657,7 +657,7 @@
});
if (pe) return pe->p_proto;
pushSTACK(NIL); /* no PLACE */
- pushSTACK(TheSubr(subr_self)->name); pushSTACK(proto);
+ pushSTACK(proto); pushSTACK(TheSubr(subr_self)->name);
check_value(error_condition,GETTEXT("~S: invalid protocol name ~S"));
proto = value1;
goto get_socket_protocol_restart;

------------------------------

Message: 3
Date: Wed, 02 Dec 2009 21:42:06 +0000
From: Sam Steingold <sds@...>
Subject: clisp/src ChangeLog,1.7216,1.7217
To: clisp-cvs@...
Message-ID: <E1NFwxa-0005U2-8v@...>

Update of /cvsroot/clisp/clisp/src
In directory sfp-cvsdas-2.v30.ch3.sourceforge.com:/tmp/cvs-serv21002/src

Modified Files:
ChangeLog
Log Message:
(get_socket_protocol) [HAVE_GETPROTOBYNAME]: fix argument order in the error message

Index: ChangeLog
===================================================================
RCS file: /cvsroot/clisp/clisp/src/ChangeLog,v
retrieving revision 1.7216
retrieving revision 1.7217
diff -u -d -r1.7216 -r1.7217
--- ChangeLog 1 Dec 2009 18:39:28 -0000 1.7216
+++ ChangeLog 2 Dec 2009 21:42:03 -0000 1.7217
@@ -1,3 +1,8 @@
+2009-12-02 Sam Steingold <sds@...>
+
+ * modules/rawsock/rawsock.c (get_socket_protocol)
+ [HAVE_GETPROTOBYNAME]: fix argument order in the error message
+
2009-12-01 Sam Steingold <sds@...>

* modules/clx/new-clx/clx.f, modules/rawsock/rawsock.c (unused):

------------------------------

Message: 4
Date: Wed, 02 Dec 2009 21:48:58 +0000
From: Sam Steingold <sds@...>
Subject: clisp/modules/rawsock test.tst,1.58,1.59
To: clisp-cvs@...
Message-ID: <E1NFx4E-0005sM-AZ@...>

Update of /cvsroot/clisp/clisp/modules/rawsock
In directory sfp-cvsdas-2.v30.ch3.sourceforge.com:/tmp/cvs-serv22581

Modified Files:
test.tst
Log Message:
add a test for check_socket_protocol

Index: test.tst
===================================================================
RCS file: /cvsroot/clisp/clisp/modules/rawsock/test.tst,v
retrieving revision 1.58
retrieving revision 1.59
diff -u -d -r1.58 -r1.59
--- test.tst 8 Oct 2009 18:58:39 -0000 1.58
+++ test.tst 2 Dec 2009 21:48:56 -0000 1.59
@@ -106,11 +106,15 @@

(catch 'type-error-handler
(handler-bind ((type-error #'type-error-handler))
- (rawsock:socket :INET :FOO nil)))
+ (rawsock:socket :INET t nil)))
NIL
(catch 'type-error-handler
(handler-bind ((type-error #'type-error-handler))
- (rawsock:socket :FOO :STREAM nil)))
+ (rawsock:socket t :STREAM nil)))
+NIL
+(catch 'type-error-handler
+ (handler-bind ((type-error #'type-error-handler))
+ (rawsock:socket :INET :STREAM t)))
NIL

(integerp (show (setq *sock* (rawsock:socket :INET :STREAM nil)))) T

------------------------------

Message: 5
Date: Wed, 02 Dec 2009 23:00:46 +0000
From: Sam Steingold <sds@...>
Subject: clisp/modules/rawsock/demos - New directory
To: clisp-cvs@...
Message-ID: <E1NFyBi-0001tI-9k@...>

Update of /cvsroot/clisp/clisp/modules/rawsock/demos
In directory sfp-cvsdas-2.v30.ch3.sourceforge.com:/tmp/cvs-serv7228/demos

Log Message:
Directory /cvsroot/clisp/clisp/modules/rawsock/demos added to the repository

------------------------------

Message: 6
Date: Wed, 02 Dec 2009 23:03:14 +0000
From: Sam Steingold <sds@...>
Subject: clisp/modules/rawsock/demos README,NONE,1.1
sniffer.lisp,NONE,1.1
To: clisp-cvs@...
Message-ID: <E1NFyE6-000243-1N@...>

Update of /cvsroot/clisp/clisp/modules/rawsock/demos
In directory sfp-cvsdas-2.v30.ch3.sourceforge.com:/tmp/cvs-serv7919/modules/rawsock/demos

Added Files:
README sniffer.lisp
Log Message:
* modules/rawsock/demos/sniffer.lisp: add a demo
* modules/rawsock/Makefile.in (clisp-module-distrib): install it

--- NEW FILE: README ---
These are various Rawsock demos.
Most have to be run as a superuser, so beware!
We assume no responsibility for your use of this software.

--- NEW FILE: sniffer.lisp ---
;;; Packet sniffer
;; run as root like this:
;; sudo clisp sniffer.lisp inet 100

(require "rawsock")

(defvar *buffer-len* 1518)

(defun my-rcvfrom (socket buffer device)
(setf (fill-pointer buffer) 1518)
(let ((len (rawsock:recvfrom socket buffer device)))
(setf (fill-pointer buffer) len)))

(defun print-buffer (buffer)
(format t "len=~:D" (length buffer))
(loop :for byte :across buffer :do (format t " ~2,'0X" byte))
(terpri))

(defun sniff (socket repeat)
(loop
:with buffer = (make-array *buffer-len* :element-type '(unsigned-byte 8)
:fill-pointer 0)
:and device = (rawsock:make-sockaddr :UNSPEC)
:repeat repeat :do
(my-rcvfrom socket buffer device)
(format t "~%family ~a " (rawsock:sockaddr-family device))
(loop :for c :across (rawsock:sockaddr-data device)
:do (format t "~c" (if (= c 0) #\space (code-char c ))))
(print-buffer buffer)))

(unless (= 2 (length *args*))
(error "Expected: ({inet|packet} repeat-count), got ~S" *args*))

(defparameter *socket*
(let ((arg (pop *args*)))
(cond ((string= arg "inet")
(rawsock:socket :inet :packet #x300))
((string= arg "packet")
(rawsock:socket :packet :raw #x300))
(t (error "invalid socket argument ~S" arg)))))
(unless (plusp *socket*)
(error "Invalid socket created for ~S: ~S" *args* *socket*))

(sniff *socket* (parse-integer (pop *args*)))

------------------------------

Message: 7
Date: Wed, 02 Dec 2009 23:03:13 +0000
From: Sam Steingold <sds@...>
Subject: clisp/modules/rawsock Makefile.in,1.13,1.14
To: clisp-cvs@...
Message-ID: <E1NFyE6-000244-1M@...>

Update of /cvsroot/clisp/clisp/modules/rawsock
In directory sfp-cvsdas-2.v30.ch3.sourceforge.com:/tmp/cvs-serv7919/modules/rawsock

Modified Files:
Makefile.in
Log Message:
* modules/rawsock/demos/sniffer.lisp: add a demo
* modules/rawsock/Makefile.in (clisp-module-distrib): install it

Index: Makefile.in
===================================================================
RCS file: /cvsroot/clisp/clisp/modules/rawsock/Makefile.in,v
retrieving revision 1.13
retrieving revision 1.14
diff -u -d -r1.13 -r1.14
--- Makefile.in 3 Sep 2009 15:28:58 -0000 1.13
+++ Makefile.in 2 Dec 2009 23:03:11 -0000 1.14
@@ -33,6 +33,8 @@

preload.lisp : $(srcdir)/preload.lisp
$(LN_S) $(srcdir)/preload.lisp .
+ mkdir -p $(distribdir)/demos
+ $(LN) $(srcdir)/demos/*.lisp $(srcdir)/demos/README $(distribdir)/demos/

# Make a module distribution into $(distribdir)
clisp-module-distrib : clisp-module force

------------------------------

Message: 8
Date: Wed, 02 Dec 2009 23:03:15 +0000
From: Sam Steingold <sds@...>
Subject: clisp/src ChangeLog,1.7217,1.7218
To: clisp-cvs@...
Message-ID: <E1NFyE7-00024E-Jl@...>

Update of /cvsroot/clisp/clisp/src
In directory sfp-cvsdas-2.v30.ch3.sourceforge.com:/tmp/cvs-serv7919/src

Modified Files:
ChangeLog
Log Message:
* modules/rawsock/demos/sniffer.lisp: add a demo
* modules/rawsock/Makefile.in (clisp-module-distrib): install it

Index: ChangeLog
===================================================================
RCS file: /cvsroot/clisp/clisp/src/ChangeLog,v
retrieving revision 1.7217
retrieving revision 1.7218
diff -u -d -r1.7217 -r1.7218
--- ChangeLog 2 Dec 2009 21:42:03 -0000 1.7217
+++ ChangeLog 2 Dec 2009 23:03:11 -0000 1.7218
@@ -1,5 +1,10 @@
2009-12-02 Sam Steingold <sds@...>

+ * modules/rawsock/demos/sniffer.lisp: add a demo
+ * modules/rawsock/Makefile.in (clisp-module-distrib): install it
+
+2009-12-02 Sam Steingold <sds@...>
+
* modules/rawsock/rawsock.c (get_socket_protocol)
[HAVE_GETPROTOBYNAME]: fix argument order in the error message

------------------------------

Message: 9
Date: Wed, 02 Dec 2009 23:17:25 +0000
From: Sam Steingold <sds@...>
Subject: clisp/modules/rawsock rawsock.xml,1.61,1.62
To: clisp-cvs@...
Message-ID: <E1NFyRp-0002qS-6u@...>

Update of /cvsroot/clisp/clisp/modules/rawsock
In directory sfp-cvsdas-2.v30.ch3.sourceforge.com:/tmp/cvs-serv10930

Modified Files:
rawsock.xml
Log Message:
mention modules/rawsock/demos/

Index: rawsock.xml
===================================================================
RCS file: /cvsroot/clisp/clisp/modules/rawsock/rawsock.xml,v
retrieving revision 1.61
retrieving revision 1.62
diff -u -d -r1.61 -r1.62
--- rawsock.xml 2 Dec 2009 21:05:59 -0000 1.61
+++ rawsock.xml 2 Dec 2009 23:17:23 -0000 1.62
@@ -29,8 +29,9 @@
so you <emphasis>still</emphasis> have to &close; the original raw socket.
</simpara></warning>

-<para>File <filename role="clisp-cvs">modules/rawsock/test.tst</filename>
- contains plenty of examples.</para>
+<para>Test file <filename role="clisp-cvs">modules/rawsock/test.tst</filename>
+ and the demos in <filename role="clisp-cvs">modules/rawsock/demos/</filename>
+ contain plenty of examples.</para>
</section>

<section id="rawsock-func"><title>Single System Call Functions</title>

------------------------------

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev

------------------------------

_______________________________________________
clisp-cvs mailing list
clisp-cvs@...
https://lists.sourceforge.net/lists/listinfo/clisp-cvs

End of clisp-cvs Digest, Vol 44, Issue 2
****************************************

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-devel mailing list
clisp-devel@...
https://lists.sourceforge.net/lists/listinfo/clisp-devel

From forum: clisp-devel

Re: Doing everything with bytes instead of chars

2009-12-02T15:50:02Z

Fred Cohen wrote:
> Fails:
>
> >sudo clisp sniffer.lisp inet 100
> *** - LOAD: A file with name rawsock does not exist

either pass "-K full" to your clisp, or install a newer clisp (with dynamic
module loading)

> On Dec 2, 2009, at 3:17 PM, Sam Steingold wrote:
>
>> thanks, please take a look at
>> http://clisp.cvs.sourceforge.net/*checkout*/clisp/clisp/modules/rawsock/demos/sniffer.lisp

contributions are welcome.

e.g.: ntp client; dns client

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: Doing everything with bytes instead of chars

2009-12-02T15:38:47Z

Fails:

>sudo clisp sniffer.lisp inet 100
*** - LOAD: A file with name rawsock does not exist

On Dec 2, 2009, at 3:17 PM, Sam Steingold wrote:

> thanks, please take a look at
> http://clisp.cvs.sourceforge.net/*checkout*/clisp/clisp/modules/rawsock/demos/sniffer.lisp
>
> ------------------------------------------------------------------------------
> Join us December 9, 2009 for the Red Hat Virtual Experience,
> a free event focused on virtualization and cloud computing.
> Attend in-depth sessions from your desk. Your couch. Anywhere.
> http://p.sf.net/sfu/redhat-sfdev2dev
> _______________________________________________
> clisp-list mailing list
> clisp-list@...
> https://lists.sourceforge.net/lists/listinfo/clisp-list
>
>

From forum: clisp-list

Re: Doing everything with bytes instead of chars

2009-12-02T15:17:55Z

thanks, please take a look at
http://clisp.cvs.sourceforge.net/*checkout*/clisp/clisp/modules/rawsock/demos/sniffer.lisp

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: more on raw sockets

2009-12-02T14:03:37Z

Don Cohen wrote:
> It now occurs to me that you have implemented much more of the socket
> interface than what I would have called "raw" sockets. It probably is
> possible to use the non-raw parts to write a version of ping that does
> not require root. Perhaps this module should be renamed to something
> more general than "raw".

they are "raw" as opposed to "socket streams", i.e., as close to the original
bsd syscalls as possible.

> Have I ever mentioned before this issue?
> (rawsock:socket :inet :packet #+ignore :all #x300)
> :all seems to be 3, which doesn't work for me, whereas #x300 does.
> Where did you get these constants? I suspect some of the others
> also.

yes, you have complained about this before.
:ETH-P-ALL is 3, as you can see by doing

(RAWSOCK:SOCKET :INET :STREAM T)

*** - RAWSOCK:SOCKET: Lisp value T is not found in table
"check_socket_protocol":
((0 :IPPROTO-IP) (41 :IPPROTO-IPV6) (1 :IPPROTO-ICMP) (255 :IPPROTO-RAW)
(6 :IPPROTO-TCP) (17 :IPPROTO-UDP) (2 :IPPROTO-IGMP) (4 :IPPROTO-IPIP)
(8 :IPPROTO-EGP) (12 :IPPROTO-PUP) (22 :IPPROTO-IDP)
(0 :IPPROTO-HOPOPTS) (43 :IPPROTO-ROUTING) (44 :IPPROTO-FRAGMENT)
(50 :IPPROTO-ESP) (51 :IPPROTO-AH) (58 :IPPROTO-ICMPV6)
(60 :IPPROTO-DSTOPTS) (59 :IPPROTO-NONE) (46 :IPPROTO-RSVP)
(47 :IPPROTO-GRE) (103 :IPPROTO-PIM) (108 :IPPROTO-COMP)
(96 :ETH-P-LOOP) (512 :ETH-P-PUP) (513 :ETH-P-PUPAT) (2048 :ETH-P-IP)
(2053 :ETH-P-X25) (2054 :ETH-P-ARP) (2303 :ETH-P-BPQ)
(2560 :ETH-P-IEEEPUP) (2561 :ETH-P-IEEEPUPAT) (24576 :ETH-P-DEC)
(24577 :ETH-P-DNA-DL) (24578 :ETH-P-DNA-RC) (24579 :ETH-P-DNA-RT)
(24580 :ETH-P-LAT) (24581 :ETH-P-DIAG) (24582 :ETH-P-CUST)
(24583 :ETH-P-SCA) (32821 :ETH-P-RARP) (32923 :ETH-P-ATALK)
(33011 :ETH-P-AARP) (33079 :ETH-P-IPX) (34525 :ETH-P-IPV6)
(34915 :ETH-P-PPP-DISC) (34916 :ETH-P-PPP-SES) (34892 :ETH-P-ATMMPOA)
(34948 :ETH-P-ATMFATE) (1 :ETH-P-802-3) (2 :ETH-P-AX25) (3 :ETH-P-ALL)
(4 :ETH-P-802-2) (5 :ETH-P-SNAP) (6 :ETH-P-DDCMP) (7 :ETH-P-WAN-PPP)
(8 :ETH-P-PPP-MP) (9 :ETH-P-LOCALTALK) (16 :ETH-P-PPPTALK)
(17 :ETH-P-TR-802-2) (21 :ETH-P-MOBITEX) (22 :ETH-P-CONTROL)
(23 :ETH-P-IRDA) (24 :ETH-P-ECONET))

these numbers come from
/usr/include/linux/if_ether.h:#define ETH_P_ALL 0x0003 /* Every packet
(be careful!!!) */

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: Doing everything with bytes instead of chars

2009-12-02T13:52:22Z

Sam Steingold writes:
> 1. (setf socket (rawsock:socket :inet :RAW 0))
> *** - UNIX error 93 (EPROTONOSUPPORT): Protocol not supported
> so I used :eth-p-all instead
Ok, since you're trying this I send my version.

> 2. (setf socket (rawsock:socket :inet :RAW :eth-p-all))
> *** - UNIX error 1 (EPERM): Operation not permitted
> so I has to do sudo.
As root won't help - my #x300 is the answer.
Try loading this:
====
(setf socket (rawsock:socket :inet :packet #+ignore :all #x300))

#+ignore ; alternative version to try
(setf socket (rawsock:socket :packet :raw #x300))

(setf device (rawsock:make-sockaddr :UNSPEC))

(setf buffer (make-array 1518 :element-type '(unsigned-byte 8) :fill-pointer 0))

(defun getraw ()
(if (not (= -1 socket))
(setf (fill-pointer buffer) 1518
len (rawsock:recvfrom socket buffer device)
(fill-pointer buffer) len)))

(defun print-buffer ()
(loop for i below len do (format t "~2,'0X " (aref buffer i))))

;; 9 show packets

(loop for i below 9 do
(getraw)
(format t "~%family ~a " (rawsock:sockaddr-family device))
(loop for i below (length (rawsock:sockaddr-data device))
as c = (aref (rawsock:sockaddr-data device) i) do
(format t "~c" (if (= c 0) #\space (code-char c ))))
(terpri)(print-buffer)(terpri))
====

If it just sits there try sending some network traffic.
Then you can try the alternative socket (uncomment 2nd expression).

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: Doing everything with bytes => rawsock

2009-12-02T13:44:18Z

Sam Steingold writes:
Oh, good - this is the message I should have replied to before.
So I'll copy the relevant parts of the other message
with subject: "[clisp-list] more on raw sockets"
to here.

> Don Cohen wrote:
> > For instance, you could provide a udp socket interface and there might
> > be some demand for it among non-root users. I don't even know what
> > other higher level facilities are available to non-root users.
>
> could you please be more specific?
> what is missing?

I now suspect that you do have all you need:
It now occurs to me that you have implemented much more of the socket
interface than what I would have called "raw" sockets. It probably is
possible to use the non-raw parts to write a version of ping that does
not require root. Perhaps this module should be renamed to something
more general than "raw".

> > How does os:resolve-host-ipaddr find the dns server?
> it calls gethostbyname or gethostbyaddr.
> http://clisp.cons.org/impnotes/syscalls.html#resolve-host
Ok, just another system call that does all of the communication
without us ever having to figure out how.

> > Let's see if this helps to clarify the problem.
> > For every packet you want to send you have to decide on an interface
> > (device) that will send it. You also have to supply the ethernet
> > header. This contains a from and to MAC address and two more bytes
> > which for ip packets will be 0800.
> > I don't know how to do any of this on non-linux machines.
> (RAWSOCK:IF-NAME-INDEX)
> (RAWSOCK:IFADDRS)
> http://clisp.cons.org/impnotes/rawsock.html#rawsock-high-level
Well, that's interesting, but I'm having a hard time figuring out
what the results mean.
For me the relevant one seems to be
#<RAWSOCK:IFADDRS :NAME "wlan0"
:FLAGS (:UP :BROADCAST :RUNNING :MULTICAST 65536)
:ADDRESS #<RAWSOCK:SOCKADDR :%DATA #(2 0 0 0 10 0 2 100 0 0 0 0 0 0 0 0)>
I gather the first 2 0 corresponds to (2 :INET)
The 10 0 2 100 is the ip address of this interface
I wonder why it happens to be in the position above. I'd have
expected either at the left or right end.
:NETMASK #<RAWSOCK:SOCKADDR :%DATA #(2 0 0 0 255 255 255 0 0 0 0 0 0 0 0 0)>
:DESTINATION
#<RAWSOCK:SOCKADDR :%DATA #(2 0 0 0 10 0 2 255 0 0 0 0 0 0 0 0)>
This is the corresponding broadcast address.
:DATA NIL>
Nothing above tells us the mac address or any reason to choose this
interface rather than any of the others.

However I now notice that the data does seem related to a different
question I asked:
When I use (rawsock:socket :packet :raw #x300) instead I get back
similar data in the buffer but some other strange data in the address
object, such as
#<RAWSOCK:SOCKADDR :%DATA #(17 0 8 0 3 0 0 0 33 3 4 6 0 33 107 64)>
In this case the data above seem to correspond to
#<RAWSOCK:IFADDRS :NAME "wmaster0" :FLAGS (:UP :RUNNING 65536)
:ADDRESS #<RAWSOCK:SOCKADDR :%DATA #(17 0 0 0 3 0 0 0 33 3 0 6 0 33 107 64)>
:NETMASK NIL :DESTINATION NIL :DATA #<FOREIGN-POINTER #x00007F4808015E88>>
I don't know enough about wmaster0 to begin to understand the address
above - what I know about it is that it does not follow the expected
frame format, which was giving me trouble.
But if I look at this result of IFADDRS
#<RAWSOCK:IFADDRS :NAME "wlan0"
:FLAGS (:UP :BROADCAST :RUNNING :MULTICAST 65536)
:ADDRESS #<RAWSOCK:SOCKADDR :%DATA #(17 0 0 0 4 0 0 0 1 0 0 6 0 33 107 64)>
The data above does correspond to other data I saw in the address
object. The mac address of that interface is 00:21:6B:40:06:7E, or
in decimal 0 33 107 64 6 126 -- suspiciously similar but not the same
as the data above.
:NETMASK NIL
:DESTINATION
#<RAWSOCK:SOCKADDR :%DATA #(17 0 0 0 4 0 0 0 1 0 0 6 255 255 255 255)>
:DATA #<FOREIGN-POINTER #x00007F4808015EE4>>

And this gives some hint about the another part of the question:
(rawsock:socket :inet :packet #+ignore :all #x300)
When I call (rawsock:sockaddr-family device) I get back UNIX on
packets that clearly came over the network.
The results in this case are actually
1 0 (which I gather means UNIX)
and then the character codes for "wlan0" followed by nulls.
So perhaps the address data for address family unix is the character
name of the interface. But it's not at all clear why using the
AF :inet to create the socket should give addresses of family UNIX when
using AF :packet to create the socket gives addresses of family PACKET.

> > On linux you can normally get a list of the available devices with
> > ifconfig. I would prefer to use the ip command (see below) but my
> > impression is that it is not as universally available as ifconfig.
> >
> > As a default, at least for addresses out in the internet,
> > I'd suggest using (if available)
> > ip route |grep default
> > which would tell you the default device
> > in combination with
> > ip addr
> > which would tell you its ip and mac address and
> > ip nei
> > which would tell you the destination mac address to use.
> > There are probably equivalent ways of getting that info from /proc but
> > I don't think those are any more universal.

> universal or not, parsing the output of a shell command is not what
> the rawsock module should do.
> do you need an interface to netlink & netdevice?
> http://linux.about.com/library/cmd/blcmdl7_netdevice.htm
> http://linux.about.com/library/cmd/blcmdl7_netlink.htm
Why are these better than shell commands?
Do we already have ways to use them from lisp?
Perhaps the answer is that we can do that from rawsock, but not in
order to figure out what data to pass in to rawsock!

> > Another approach to filling in the same info would be to read packets
> this appear to involve catching the packets (frames) that gethostbyname(2)
> generates, which, I guess, requires running as root.
Right, but when I wrote that I was imagining that we were using raw
sockets rather than the more general c socket interface.

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: Doing everything with bytes instead of chars

2009-12-02T13:38:08Z

On Dec 2, 2009, at 1:19 PM, Sam Steingold wrote:

> Fred Cohen wrote:
>> (IN-PACKAGE "RAWSOCK")
>> (setf socket (rawsock:socket :inet :RAW 0 #+ignore :all)
>> device
>> (rawsock:make-sockaddr :UNSPEC
>> (make-array 14 :element-type '(unsigned-byte 8)))
>> buffer
>> (make-array 1518 :element-type '(unsigned-byte 8) :fill-pointer
>> 1518)))
>> (configdev socket "eth0" 1 1 ) ; eth0 no arp raw mode on socket
>> (defun getraw ()
>> (if (not (= -1 socket))
>> (setf (fill-pointer buffer) 1518
>> len #+ignore (rawsock::recvfr socket)
>> (rawsock:recvfrom socket buffer device)
>> (fill-pointer buffer) len)))
>> (defun print-buffer () (loop for i below len do (format t "~2,'0X
>> " (aref buffer i))))
>> ;; a rudimentary dump of all packets seen.
>> (loop while true do ((getraw)(print-buffer)))
>
>
> 1. (setf socket (rawsock:socket :inet :RAW 0))
>
> *** - UNIX error 93 (EPROTONOSUPPORT): Protocol not supported
>
> so I used :eth-p-all instead
>
> 2. (setf socket (rawsock:socket :inet :RAW :eth-p-all))
>
> *** - UNIX error 1 (EPERM): Operation not permitted
>
> so I has to do sudo.

Yes - you will have to run as root no matter what for raw sockets to
work.

> 3. with the obvious syntax fixes, the last look just sits there
> without printing anything.

Right - till the socket gets something, nothing will print out. You
need to send it something...

>>> I mean the address argument is expected to be an int32.
>>> Not even a string is accepted.
>>> It would seem, btw, that the IP address argument should actually
>>> be a socket address, as returned by host->ip in rawsock/test.tst.
>> IPv4 addresses are 4 bytes (octets) long - an int32.

This is about shoving bytes in and out of interfaces. IP addresses are
never stings...

> so, why configdev cannot handle IPv6 addresses?

It probably could, but I haven't had time to go through all of the
possible uses for it.

>>>> Nothing to fix.
>>> I thought you said that configdev modifies the behavior of both
>>> the socket and the device (e.g., eth0).
>>> I can close the socket and open a new one, but how do I bring the
>>> device back to the original behavior?
>> (defun close-sock ()
>> (if (= (rawsock:sock-close #+ignore rawsock::closesock socket) -1)
>> (error "can't close socket")))
>
> you mean this resets the device as well?

It removes the socket handler from the handlers list in the kernel.
There are permanent (OS-wide) changes and by-handler changes that
happen to the process of handling packets. Changes that apply to all
handlers are not changed by the close, but all handling done by the
handler is no longer done, so for example, if there is an IP handler
still running (e.g., a TCP handler), it might still operate after
removing the raw handler (and may even run while the raw handler is
running... talk about dangerous)!

As a rule of thumb, I would say you should redo any ifconfig you have
for an interface after you are no longer using raw sockets on it, and
not try to use both raw sockets and regular IP traffic handling on the
same interface. It is likely that you will have unpredictable and
possible catastrophic results (to your mechanisms and others).

Having said that, I have, from time to time, decided to observe such
results for scientific purposes...

FC
- This communication is confidential to the parties it is intended to
serve -
Fred Cohen & Associates tel/fax: 925-454-0171
http://all.net/ 572 Leona Drive Livermore, CA 94550
Join http://tech.groups.yahoo.com/group/FCA-announce/join for our
mailing list

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: Doing everything with bytes instead of chars

2009-12-02T13:19:57Z

Fred Cohen wrote:

1. (setf socket (rawsock:socket :inet :RAW 0))

*** - UNIX error 93 (EPROTONOSUPPORT): Protocol not supported

so I used :eth-p-all instead

2. (setf socket (rawsock:socket :inet :RAW :eth-p-all))

*** - UNIX error 1 (EPERM): Operation not permitted

so I has to do sudo.

3. with the obvious syntax fixes, the last look just sits there without
printing anything.

>> I mean the address argument is expected to be an int32.
>> Not even a string is accepted.
>> It would seem, btw, that the IP address argument should actually be
>> a socket address, as returned by host->ip in rawsock/test.tst.
>
> IPv4 addresses are 4 bytes (octets) long - an int32.

so, why configdev cannot handle IPv6 addresses?

>>> Nothing to fix.
>> I thought you said that configdev modifies the behavior of both the
>> socket and the device (e.g., eth0).
>> I can close the socket and open a new one, but how do I bring the
>> device back to the original behavior?
>
> (defun close-sock ()
> (if (= (rawsock:sock-close #+ignore rawsock::closesock socket) -1)
> (error "can't close socket")))

you mean this resets the device as well?

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: Doing everything with bytes => rawsock

2009-12-02T11:53:27Z

Don Cohen wrote:
> Then you need to provide access to other low level facilities that are
> not as capable as raw sockets and therefore require less privilege.

what system calls are missing?

> For instance, you could provide a udp socket interface and there might
> be some demand for it among non-root users. I don't even know what
> other higher level facilities are available to non-root users.

could you please be more specific?
what is missing?

> How does os:resolve-host-ipaddr find the dns server?

it calls gethostbyname or gethostbyaddr.
http://clisp.cons.org/impnotes/syscalls.html#resolve-host

> Let's see if this helps to clarify the problem.
> For every packet you want to send you have to decide on an interface
> (device) that will send it. You also have to supply the ethernet
> header. This contains a from and to MAC address and two more bytes
> which for ip packets will be 0800.
> I don't know how to do any of this on non-linux machines.

(RAWSOCK:IF-NAME-INDEX)
(RAWSOCK:IFADDRS)
http://clisp.cons.org/impnotes/rawsock.html#rawsock-high-level

> On linux you can normally get a list of the available devices with
> ifconfig. I would prefer to use the ip command (see below) but my
> impression is that it is not as universally available as ifconfig.
>
> As a default, at least for addresses out in the internet,
> I'd suggest using (if available)
> ip route |grep default
> which would tell you the default device
> in combination with
> ip addr
> which would tell you its ip and mac address and
> ip nei
> which would tell you the destination mac address to use.
> There are probably equivalent ways of getting that info from /proc but
> I don't think those are any more universal.

universal or not, parsing the output of a shell command is not what the rawsock
module should do.
do you need an interface to netlink & netdevice?
http://linux.about.com/library/cmd/blcmdl7_netdevice.htm
http://linux.about.com/library/cmd/blcmdl7_netlink.htm

> Another approach to filling in the same info would be to read packets
> coming in from the internet, perhaps in response to a command that you
> execute. For instance, if you do an os:resolve-host-ipaddr you can
> expect to see a dns reply which will tell you among other things
> - the ip address of the dns server
> - the interface used to reach it
> - the mac addresses you'd need to send packets out that interface
> - your ip address (the one to which the reply was sent)

this appear to involve catching the packets (frames) that gethostbyname(2)
generates, which, I guess, requires running as root.

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

more on raw sockets

2009-12-02T10:58:44Z

Sorry about not continuing the previous thread but I've already
deleted the messages.

It now occurs to me that you have implemented much more of the socket
interface than what I would have called "raw" sockets. It probably is
possible to use the non-raw parts to write a version of ping that does
not require root. Perhaps this module should be renamed to something
more general than "raw".

Have I ever mentioned before this issue?
(rawsock:socket :inet :packet #+ignore :all #x300)
:all seems to be 3, which doesn't work for me, whereas #x300 does.
Where did you get these constants? I suspect some of the others
also.
When I call (rawsock:sockaddr-family device) I get back UNIX on
packets that clearly came over the network. I also see it returning
801 though I don't see where that is defined.

Another question: When I use the call above with :inet I get back
reasonable looking results from rawsock:sockaddr-data - things that
look like the device names shown by ifconfig.
When I use (rawsock:socket :packet :raw #x300) instead I get back
similar data in the buffer but some other strange data in the address
object, such as
#<RAWSOCK:SOCKADDR :%DATA #(17 0 8 0 3 0 0 0 33 3 4 6 0 33 107 64)>
This line in man socket suggests that :packet is what I want.
AF_PACKET Low level packet interface packet(7)
Any ideas?

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

clisp-cvs Digest, Vol 44, Issue 1

2009-12-02T04:04:26Z

Send clisp-cvs mailing list submissions to
clisp-cvs@...

To subscribe or unsubscribe via the World Wide Web, visit
https://lists.sourceforge.net/lists/listinfo/clisp-cvs
or, via email, send a message with subject or body 'help' to
clisp-cvs-request@...

You can reach the person managing the list at
clisp-cvs-owner@...

When replying, please edit your Subject line so it is more specific
than "Re: Contents of clisp-cvs digest..."

CLISP CVS commits for today

Today's Topics:

1. clisp/src ChangeLog,1.7214,1.7215 (Sam Steingold)
2. clisp/modules/rawsock rawsock.c,1.124,1.125 (Sam Steingold)
3. clisp/doc common.xsl,1.77,1.78 (Sam Steingold)
4. clisp/modules/rawsock rawsock.c,1.125,1.126 (Sam Steingold)
5. clisp/modules/clx/new-clx clx.f,2.178,2.179 (Sam Steingold)
6. clisp/src ChangeLog,1.7215,1.7216 (Sam Steingold)
7. clisp/modules/rawsock rawsock.xml,1.58,1.59 (Sam Steingold)
8. clisp/modules/rawsock rawsock.xml,1.59,1.60 (Sam Steingold)
9. clisp/doc impent.xml,1.361,1.362 (Sam Steingold)

----------------------------------------------------------------------

Message: 1
Date: Tue, 01 Dec 2009 18:34:22 +0000
From: Sam Steingold <sds@...>
Subject: clisp/src ChangeLog,1.7214,1.7215
To: clisp-cvs@...
Message-ID: <E1NFXYN-0007rS-1n@...>

Update of /cvsroot/clisp/clisp/src
In directory sfp-cvsdas-2.v30.ch3.sourceforge.com:/tmp/cvs-serv30172/src

Modified Files:
ChangeLog
Log Message:
(configdev): use strncpy/IFNAMSIZ
instead of strcpy for copying the interface name

Index: ChangeLog
===================================================================
RCS file: /cvsroot/clisp/clisp/src/ChangeLog,v
retrieving revision 1.7214
retrieving revision 1.7215
diff -u -d -r1.7214 -r1.7215
--- ChangeLog 30 Nov 2009 04:30:27 -0000 1.7214
+++ ChangeLog 1 Dec 2009 18:34:20 -0000 1.7215
@@ -1,3 +1,8 @@
+2009-12-01 Sam Steingold <sds@...>
+
+ * modules/rawsock/rawsock.c (configdev): use strncpy/IFNAMSIZ
+ instead of strcpy for copying the interface name
+
2009-11-29 Sam Steingold <sds@...>

* m4/clisp.m4 (CL_CLISP): use "." instead of "source" (bash-specific)

------------------------------

Message: 2
Date: Tue, 01 Dec 2009 18:34:22 +0000
From: Sam Steingold <sds@...>
Subject: clisp/modules/rawsock rawsock.c,1.124,1.125
To: clisp-cvs@...
Message-ID: <E1NFXYM-0007rM-Ka@...>

Update of /cvsroot/clisp/clisp/modules/rawsock
In directory sfp-cvsdas-2.v30.ch3.sourceforge.com:/tmp/cvs-serv30172/modules/rawsock

Modified Files:
rawsock.c
Log Message:
(configdev): use strncpy/IFNAMSIZ
instead of strcpy for copying the interface name

Index: rawsock.c
===================================================================
RCS file: /cvsroot/clisp/clisp/modules/rawsock/rawsock.c,v
retrieving revision 1.124
retrieving revision 1.125
diff -u -d -r1.124 -r1.125
--- rawsock.c 12 Nov 2009 20:36:13 -0000 1.124
+++ rawsock.c 1 Dec 2009 18:34:20 -0000 1.125
@@ -1145,12 +1145,12 @@
}

#if defined(HAVE_NET_IF_H)
-/* STACK_1 = name, for error reporting */
-static void configdev (rawsock_t sock, char* name, int ipaddress, int flags) {
+/* STACK_1 = ifname, for error reporting */
+static void configdev (rawsock_t sock, char* ifname, int ipaddress, int flags) {
struct ifreq ifrequest;
#if defined(SIOCGIFFLAGS) && defined(SIOCSIFFLAGS)
memset(&ifrequest, 0, sizeof(struct ifreq));
- strcpy(ifrequest.ifr_name, name);
+ strncpy(ifrequest.ifr_name, ifname, IFNAMSIZ);
if (ioctl(sock, SIOCGIFFLAGS, &ifrequest) < 0)
OS_file_error(STACK_1);
ifrequest.ifr_flags |= flags;
@@ -1159,7 +1159,7 @@
#endif
#if defined(SIOCGIFADDR) && defined(SIOCSIFADDR)
memset(&ifrequest, 0, sizeof(struct ifreq));
- strcpy(ifrequest.ifr_name, name);
+ strncpy(ifrequest.ifr_name, ifname, IFNAMSIZ);
if (ioctl(sock, SIOCGIFADDR, &ifrequest) < 0)
OS_file_error(STACK_1);
/* address was 0.0.0.0 -> error */
@@ -1177,13 +1177,13 @@
}

DEFFLAGSET(configdev_flags,IFF_PROMISC IFF_NOARP)
-DEFUN(RAWSOCK:CONFIGDEV, socket name ipaddress &key PROMISC NOARP) {
+DEFUN(RAWSOCK:CONFIGDEV, socket ifname ipaddress &key PROMISC NOARP) {
int flags = configdev_flags();
uint32 ipaddress = I_to_UL(check_uint32(STACK_0));
rawsock_t sock = I_to_uint(check_uint(STACK_2));
- with_string_0(check_string(STACK_1),Symbol_value(S(utf_8)),name, {
+ with_string_0(check_string(STACK_1),Symbol_value(S(utf_8)),ifname, {
begin_blocking_system_call();
- configdev(sock, name, ipaddress, flags);
+ configdev(sock, ifname, ipaddress, flags);
end_blocking_system_call();
});
VALUES0; skipSTACK(3);

------------------------------

Message: 3
Date: Tue, 01 Dec 2009 18:35:46 +0000
From: Sam Steingold <sds@...>
Subject: clisp/doc common.xsl,1.77,1.78
To: clisp-cvs@...
Message-ID: <E1NFXZi-00087P-47@...>

Update of /cvsroot/clisp/clisp/doc
In directory sfp-cvsdas-2.v30.ch3.sourceforge.com:/tmp/cvs-serv31185

Modified Files:
common.xsl
Log Message:
RFC: rfc.net seems dead, use www.ietf.org/rfc instead

Index: common.xsl
===================================================================
RCS file: /cvsroot/clisp/clisp/doc/common.xsl,v
retrieving revision 1.77
retrieving revision 1.78
diff -u -d -r1.77 -r1.78
--- common.xsl 29 Jun 2009 18:12:25 -0000 1.77
+++ common.xsl 1 Dec 2009 18:35:44 -0000 1.78
@@ -88,9 +88,9 @@
><xsl:apply-templates/></a></xsl:template>


-<xsl:param name="rfc.top" select="'http://rfc.net/rfc'"/>
+<xsl:param name="rfc.top" select="'http://www.ietf.org/rfc/rfc'"/>
<xsl:template match="ulink[@role='rfc']">
- <a class="{@role}" href="{$rfc.top}{@url}.html"><code>
+ <a class="{@role}" href="{$rfc.top}{@url}.txt"><code>
<xsl:choose><xsl:when test=".=''"><xsl:text>RFC</xsl:text>
<xsl:value-of select="@url"/></xsl:when>
<xsl:otherwise><xsl:apply-templates/></xsl:otherwise></xsl:choose>

------------------------------

Message: 4
Date: Tue, 01 Dec 2009 18:39:30 +0000
From: Sam Steingold <sds@...>
Subject: clisp/modules/rawsock rawsock.c,1.125,1.126
To: clisp-cvs@...
Message-ID: <E1NFXdK-0000Mu-1H@...>

Update of /cvsroot/clisp/clisp/modules/rawsock
In directory sfp-cvsdas-2.v30.ch3.sourceforge.com:/tmp/cvs-serv1382/modules/rawsock

Modified Files:
rawsock.c
Log Message:
(unused): define depending on __GNUC__, not GNU

Index: rawsock.c
===================================================================
RCS file: /cvsroot/clisp/clisp/modules/rawsock/rawsock.c,v
retrieving revision 1.125
retrieving revision 1.126
diff -u -d -r1.125 -r1.126
--- rawsock.c 1 Dec 2009 18:34:20 -0000 1.125
+++ rawsock.c 1 Dec 2009 18:39:27 -0000 1.126
@@ -1548,7 +1548,7 @@
VALUES1(fixnum(length)); skipSTACK(1);
}

-#ifdef GNU /* to prevent a gcc-warning "statement with no effect" */
+#ifdef __GNUC__ /* to prevent a gcc-warning "statement with no effect" */
# define unused (void)
#else
# define unused

------------------------------

Message: 5
Date: Tue, 01 Dec 2009 18:39:29 +0000
From: Sam Steingold <sds@...>
Subject: clisp/modules/clx/new-clx clx.f,2.178,2.179
To: clisp-cvs@...
Message-ID: <E1NFXdK-0000Ms-0m@...>

Update of /cvsroot/clisp/clisp/modules/clx/new-clx
In directory sfp-cvsdas-2.v30.ch3.sourceforge.com:/tmp/cvs-serv1382/modules/clx/new-clx

Modified Files:
clx.f
Log Message:
(unused): define depending on __GNUC__, not GNU

Index: clx.f
===================================================================
RCS file: /cvsroot/clisp/clisp/modules/clx/new-clx/clx.f,v
retrieving revision 2.178
retrieving revision 2.179
diff -u -d -r2.178 -r2.179
--- clx.f 12 Nov 2009 20:36:13 -0000 2.178
+++ clx.f 1 Dec 2009 18:39:27 -0000 2.179
@@ -442,7 +442,7 @@
#include <X11/extensions/shape.h>
##endif

-#ifdef GNU /* to prevent a gcc-warning "statement with no effect" */
+#ifdef __GNUC__ /* to prevent a gcc-warning "statement with no effect" */
# define unused (void)
#else
# define unused

------------------------------

Message: 6
Date: Tue, 01 Dec 2009 18:39:30 +0000
From: Sam Steingold <sds@...>
Subject: clisp/src ChangeLog,1.7215,1.7216
To: clisp-cvs@...
Message-ID: <E1NFXdK-0000N2-9A@...>

Update of /cvsroot/clisp/clisp/src
In directory sfp-cvsdas-2.v30.ch3.sourceforge.com:/tmp/cvs-serv1382/src

Modified Files:
ChangeLog
Log Message:
(unused): define depending on __GNUC__, not GNU

Index: ChangeLog
===================================================================
RCS file: /cvsroot/clisp/clisp/src/ChangeLog,v
retrieving revision 1.7215
retrieving revision 1.7216
diff -u -d -r1.7215 -r1.7216
--- ChangeLog 1 Dec 2009 18:34:20 -0000 1.7215
+++ ChangeLog 1 Dec 2009 18:39:28 -0000 1.7216
@@ -1,5 +1,10 @@
2009-12-01 Sam Steingold <sds@...>

+ * modules/clx/new-clx/clx.f, modules/rawsock/rawsock.c (unused):
+ define depending on __GNUC__, not GNU
+
+2009-12-01 Sam Steingold <sds@...>
+
* modules/rawsock/rawsock.c (configdev): use strncpy/IFNAMSIZ
instead of strcpy for copying the interface name

------------------------------

Message: 7
Date: Tue, 01 Dec 2009 19:16:39 +0000
From: Sam Steingold <sds@...>
Subject: clisp/modules/rawsock rawsock.xml,1.58,1.59
To: clisp-cvs@...
Message-ID: <E1NFYDH-0002O0-PH@...>

Update of /cvsroot/clisp/clisp/modules/rawsock
In directory sfp-cvsdas-2.v30.ch3.sourceforge.com:/tmp/cvs-serv9168

Modified Files:
rawsock.xml
Log Message:
typos

Index: rawsock.xml
===================================================================
RCS file: /cvsroot/clisp/clisp/modules/rawsock/rawsock.xml,v
retrieving revision 1.58
retrieving revision 1.59
diff -u -d -r1.58 -r1.59
--- rawsock.xml 1 Sep 2009 14:25:23 -0000 1.58
+++ rawsock.xml 1 Dec 2009 19:16:37 -0000 1.59
@@ -242,9 +242,9 @@
<varlistentry id="rawsock-if-name-index"><term><code>(RAWSOCK:IF-NAME-INDEX
&optional-amp; &what-r;)</code></term>
<listitem><simpara>Call <function role="unix">if_nametoindex</function>
- when &net-r; is a &string-t; and return an &integer-t;;
+ when &when-r; is a &string-t; and return an &integer-t;;
or call <function role="unix">if_indextoname</function> when
- &net-r; is an &integer-t; and return a &string-t;.
+ &what-r; is an &integer-t; and return a &string-t;.
When &what-r; is &nil;, return an &alist; of
pairs <literal role="data">(&index-r; . &name-r;)</literal>
using <function role="unix">if_nameindex</function>.
@@ -320,8 +320,9 @@
The typical packet you send is both &ip; and &tcp; and thus has two
checksums, so you would want to call <emphasis>two</emphasis> functions.
</simpara></listitem></varlistentry>
-<varlistentry><term><code>(RAWSOCK:CONFIGDEV &sock-r; &name-r; &addr-r;
- &key-amp; <replaceable>promisc</replaceable>
+<varlistentry><term><code>(RAWSOCK:CONFIGDEV &sock-r;
+ <replaceable>ifname</replaceable> &addr-r; &key-amp;
+ <replaceable>promisc</replaceable>
<replaceable>noarp</replaceable>)</code></term>
<listitem><simpara>Set some socket options and &ip; &addr-r;
with <function role="unix">ioctl</function>.

------------------------------

Message: 8
Date: Tue, 01 Dec 2009 22:43:45 +0000
From: Sam Steingold <sds@...>
Subject: clisp/modules/rawsock rawsock.xml,1.59,1.60
To: clisp-cvs@...
Message-ID: <E1NFbRh-0005ZI-JK@...>

Update of /cvsroot/clisp/clisp/modules/rawsock
In directory sfp-cvsdas-2.v30.ch3.sourceforge.com:/tmp/cvs-serv21392/modules/rawsock

Modified Files:
rawsock.xml
Log Message:
add &or-e;

Index: rawsock.xml
===================================================================
RCS file: /cvsroot/clisp/clisp/modules/rawsock/rawsock.xml,v
retrieving revision 1.59
retrieving revision 1.60
diff -u -d -r1.59 -r1.60
--- rawsock.xml 1 Dec 2009 19:16:37 -0000 1.59
+++ rawsock.xml 1 Dec 2009 22:43:43 -0000 1.60
@@ -256,7 +256,7 @@
filtered using flags, e.g., <code>(ifaddrs :flags-or '(a b)
:flags-and '(c d))</code> will return a list of objects which have
flags <literal>c</literal> &and-e; <literal>d</literal> &and-e; at
- least one of <literal>a</literal> or <literal>b</literal> set.
+ least one of <literal>a</literal> &or-e; <literal>b</literal> set.
</simpara></listitem></varlistentry>
<varlistentry id="rawsock-sockopt"><term><code>(RAWSOCK:SOCKET-OPTION
&sock-r; &name-r; &key-amp; :LEVEL)</code></term>

------------------------------

Message: 9
Date: Tue, 01 Dec 2009 22:43:45 +0000
From: Sam Steingold <sds@...>
Subject: clisp/doc impent.xml,1.361,1.362
To: clisp-cvs@...
Message-ID: <E1NFbRh-0005ZG-I7@...>

Update of /cvsroot/clisp/clisp/doc
In directory sfp-cvsdas-2.v30.ch3.sourceforge.com:/tmp/cvs-serv21392/doc

Modified Files:
impent.xml
Log Message:
add &or-e;

Index: impent.xml
===================================================================
RCS file: /cvsroot/clisp/clisp/doc/impent.xml,v
retrieving revision 1.361
retrieving revision 1.362
diff -u -d -r1.361 -r1.362
--- impent.xml 19 Nov 2009 21:02:22 -0000 1.361
+++ impent.xml 1 Dec 2009 22:43:43 -0000 1.362
@@ -1070,6 +1070,7 @@
<!ENTITY yes-en '<entry>yes</entry>'>
<!ENTITY no-en '<entry>no</entry>'>
<!ENTITY and-e '<emphasis role="strong">and</emphasis>'>
+<!ENTITY or-e '<emphasis role="strong">or</emphasis>'>
<!ENTITY not-e '<emphasis role="strong">not</emphasis>'>
<!ENTITY is-e '<emphasis role="strong">is</emphasis>'>
<!ENTITY after-e '<emphasis role="strong">after</emphasis>'>

------------------------------

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev

------------------------------

_______________________________________________
clisp-cvs mailing list
clisp-cvs@...
https://lists.sourceforge.net/lists/listinfo/clisp-cvs

End of clisp-cvs Digest, Vol 44, Issue 1
****************************************

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-devel mailing list
clisp-devel@...
https://lists.sourceforge.net/lists/listinfo/clisp-devel

From forum: clisp-devel

Re: Doing everything with bytes => rawsock

2009-12-01T13:35:13Z

Sam Steingold writes:

> > Although ping does not require root, an implementation of ping
> using > rawsock WOULD require root, because you would be opening a
> raw socket.
> I don't understand this, so I will take your word.
The real ping program does not create a raw socket.
The people who put together OS environments think it's reasonable to
allow non-root users to call that program.
They don't think it's reasonable for non-root users to be able to open
raw sockets because once you have one you can use it for a lot more
than ping.

> of course, I would much prefer an implementation which would not
> require root privileges. nobody in his right mind would ever run
> sudo demo
Then you need to provide access to other low level facilities that are
not as capable as raw sockets and therefore require less privilege.
For instance, you could provide a udp socket interface and there might
be some demand for it among non-root users. I don't even know what
other higher level facilities are available to non-root users.
I think that most of what rawsock users will want to do really
deserves to be limited to root.
(That's true of pretty much everything that Fred and I do with raw
sockets. Do you know of any other users?)

> we already have os:resolve-host-ipaddr.
> actually, we have had it for 10.5 years, since 1999-03-10!
Ok, though I thought maybe you'd like that as the first demo.
Maybe you can use tcp for dns. It might depend on your dns server.
How does os:resolve-host-ipaddr find the dns server? Is that also
available as a lisp function?

> > Above only begins to hint at how low level this stuff is. This is a
> > problem for demos because there are too many details for anyone to
> > want to control, but it's not clear which ones the user (demo viewer)
> > will want to control, and each one leads to its own issue of how to
> > even find out which options are available. I could write something
> > that would work out of MY box, but when it fails on your box you'll
> > have to understand a lot in order to figure out why.
>
> this is no good, of course.
> can you write a simple general purpose application, like ping or traceroute?
...
> how about a trivial ntp client instead then?

Let's see if this helps to clarify the problem.
For every packet you want to send you have to decide on an interface
(device) that will send it. You also have to supply the ethernet
header. This contains a from and to MAC address and two more bytes
which for ip packets will be 0800.
I don't know how to do any of this on non-linux machines.
On linux you can normally get a list of the available devices with
ifconfig. I would prefer to use the ip command (see below) but my
impression is that it is not as universally available as ifconfig.

As a default, at least for addresses out in the internet,
I'd suggest using (if available)
ip route |grep default
which would tell you the default device
in combination with
ip addr
which would tell you its ip and mac address and
ip nei
which would tell you the destination mac address to use.
There are probably equivalent ways of getting that info from /proc but
I don't think those are any more universal.

Another approach to filling in the same info would be to read packets
coming in from the internet, perhaps in response to a command that you
execute. For instance, if you do an os:resolve-host-ipaddr you can
expect to see a dns reply which will tell you among other things
- the ip address of the dns server
- the interface used to reach it
- the mac addresses you'd need to send packets out that interface
- your ip address (the one to which the reply was sent)
Of course, if you're unlucky, the interface you use to reach your
dns server might not be capable of reaching the host you want to ping.
On the other hand, with my solution your default interface might not
be able to reach the host you want to ping either (which is why I said
"out on the internet" above).

At this point we're ready to start filling in the IP header. If
you've looked at the rfc's you'll know there are gobs of choices
there. At least most of them don't require querying your environment.

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: Doing everything with bytes instead of chars

2009-12-01T13:26:30Z

On Dec 1, 2009, at 12:52 PM, Sam Steingold wrote:

> Fred Cohen wrote:
>> On Dec 1, 2009, at 11:08 AM, Sam Steingold wrote:
>>> Fred Cohen wrote:
>>>> configdev sets of the IO device (network card - the "dev"
>>>> argument) on
>>> "configdev sets the IO device"?
>> Yes - the interface being used to send and receive. (e.g., eth0,
>> etc.)
>
> could you please provide some code which uses this function?

(IN-PACKAGE "RAWSOCK")
(setf socket (rawsock:socket :inet :RAW 0 #+ignore :all)
device
(rawsock:make-sockaddr :UNSPEC
(make-array 14 :element-type '(unsigned-byte 8)))
buffer
(make-array 1518 :element-type '(unsigned-byte 8) :fill-pointer 1518)))
(configdev socket "eth0" 1 1 ) ; eth0 no arp raw mode on socket
(defun getraw ()
(if (not (= -1 socket))
(setf (fill-pointer buffer) 1518
len #+ignore (rawsock::recvfr socket)
(rawsock:recvfrom socket buffer device)
(fill-pointer buffer) len)))
(defun print-buffer () (loop for i below len do (format t "~2,'0X
" (aref buffer i))))
;; a rudimentary dump of all packets seen.
(loop while true do ((getraw)(print-buffer)))

>
> e.g.,
>
> (defparameter *sock* (rawsock:socket ???))
> (rawsock:configdev *sock* "eth0" ...)
> (rawsock:sock-read *sock* ...)
> or whatever.
>
>>>> the identified raw socket (scocket argument) to promiscuous
>>>> mode (accept all packets and pass them into the program) no ARP
>>>> mode (the interface does not automatically respond to ARP
>>>> requests, and configures its built-in address to 0.0.0.0 (which
>>>> is to say, it has no particular address).
>>> actually, the code only does something when address is non 0.
>>> (and it only handles ipv4, not ipv6).
>> Actually, it handles any raw bytes on an interface - the checksum
>> code is for IPV4
>
> I mean the address argument is expected to be an int32.
> Not even a string is accepted.
> It would seem, btw, that the IP address argument should actually be
> a socket address, as returned by host->ip in rawsock/test.tst.

IPv4 addresses are 4 bytes (octets) long - an int32.

>>>> Normally, if you want a silent interface which allows true
>>>> arbitrary behavior bases on what the program tells the
>>>> interface to do, you want all of these to be the case. When you
>>>> are done, you may want to turn these things back to some
>>>> previous state (i.e., the normal IP address, ARP responses per
>>>> the configured ARP address of the Ethernet card, and ignoring
>>>> non-broadcast packets sent to the interface, since this is the
>>>> normal behavior of an ethernet card and reduces system resource
>>>> consumption (if there are no promiscuous modes set by other
>>>> processes.
>>> so, we are not providing a way to restore the original behavior,
>>> right?
>> You do another configdev to set things to whatever you want them to
>> be when you don't want "raw". The row socket is not persistent or
>> in effect past the process, so other processes may do as they
>> please and handle other raw or non-raw interfaces - although
>> performance will be hit because the interface is looking at
>> everything instead of ignoring packets not broadcast or to its MAC
>> address / IP address. The OS keeps a list of interfaces and the
>> processes that get each of them at what phase in its processing. A
>> raw socket avoids the OS handling (except for the low-level driver
>> code to move bytes via the DMA to and from the hardware and
>> memory). The configdev code uses IOCTL to alter the hardware
>> configuration and tell the OS to put you at an earlier place in
>> the list of handlers.
>>> this should be fixed.
>> Nothing to fix.
>
> I thought you said that configdev modifies the behavior of both the
> socket and the device (e.g., eth0).
> I can close the socket and open a new one, but how do I bring the
> device back to the original behavior?

(defun close-sock ()
(if (= (rawsock:sock-close #+ignore rawsock::closesock socket) -1)
(error "can't close socket")))

>>>>> 2. checksum functions: what is their contract? (e.g., min/max
>>>>> buffer size? do they modify their argument? how?)
>>>>> where specifically are these checksums documented?
>>>>> how are these functions supposed to be used?
>>>> IP datagrams are limited in total size by RFC791 (http://www.ietf.org/rfc/rfc791.txt
>>> looking at page 10, I see Figure 4. (Example Internet Datagram
>>> Header).
>>> it says that the "Total Length" is in bytes 2 and 3 (0-based,
>>> i.e., bits 16-31)
>>> while "Header Checksum" is in bytes 10 and 11.
>>>
>>> code in ipcsum takes length not buffer[14] and writes the csum
>>> into bytes 25&26.
>>>
>>> I am confused.
>> That's because the first 14 bytes of an Ethernet frame are
>> (normally) for the MAC address. IP datagrams sit inside ethernet
>> frames (or other frames depending on the particulars). rawsock
>> assumes an ethernet - which is the most common situation. So
>> everything is offset from IP by 14 bytes. But ethernet frames are
>> also alterable by rawsock - so you can operate many different
>> ethernet MAC addresses. Ethernet frames include things like
>> manufacturer identification information and from and to MAC
>> addresses (and broadcast addresses), which also have to get
>> changed around to send and receive, and have to get changed when
>> forwarding if the interfaces deal with other infrastructure
>> elements (e.g., switches) that use MAC addresses for decision-
>> making.
>
> I see, so "buffer" is not a "datagram" but a "frame".

Yes.

> what can clisp do with such a frame?

Anything you want - read it, write it, load from the ethernet
interface, send to the ethernet interface, alter when in your
possession.

> can you give a code sample?

(loop while true do ((getraw)
(setf (aref buffer 14) 5)
(sendt socket len)
) )

This will change the IP version number to 5 (if it was 4, it will
cause problems...) and send the frame out with the new bytes.

>> It gets even worse because loopback (the lo interface) does not
>> have ethernet frames (it's not on an ethernet) so for these cases
>> the code has to be abused (you do the checksums than shift the
>> bytes to go from ethernet to lo, and add your own MAC addresses
>> which have to be right to lo datagrams to put then in ethernet
>> frames. rawsock does not do this for you.
>
> the csum functions now take :start/:end arguments, so, I guess, you
> can format them for loopback too.
> actually, I would love to see a piece of code for that.

As I said, I haven't looked at this for a while - and haven't seen the
new versions. It's the same thing as before - just start at 0 instead
of 14 offset.

FC
- This communication is confidential to the parties it is intended to
serve -
Fred Cohen & Associates tel/fax: 925-454-0171
http://all.net/ 572 Leona Drive Livermore, CA 94550
Join http://tech.groups.yahoo.com/group/FCA-announce/join for our
mailing list

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: Doing everything with bytes instead of chars

2009-12-01T12:52:25Z

Fred Cohen wrote:
> On Dec 1, 2009, at 11:08 AM, Sam Steingold wrote:
>
>> Fred Cohen wrote:
>>> configdev sets of the IO device (network card - the "dev" argument)
>>> on
>> "configdev sets the IO device"?
>
> Yes - the interface being used to send and receive. (e.g., eth0, etc.)

could you please provide some code which uses this function?

e.g.,

(defparameter *sock* (rawsock:socket ???))
(rawsock:configdev *sock* "eth0" ...)
(rawsock:sock-read *sock* ...)
or whatever.

>>> the identified raw socket (scocket argument) to promiscuous mode
>>> (accept all packets and pass them into the program) no ARP mode
>>> (the interface does not automatically respond to ARP requests,
>>> and configures its built-in address to 0.0.0.0 (which is to say,
>>> it has no particular address).
>> actually, the code only does something when address is non 0.
>> (and it only handles ipv4, not ipv6).
>
> Actually, it handles any raw bytes on an interface - the checksum code
> is for IPV4

I mean the address argument is expected to be an int32.
Not even a string is accepted.
It would seem, btw, that the IP address argument should actually be a socket
address, as returned by host->ip in rawsock/test.tst.

>>> Normally, if you want a silent interface which allows true
>>> arbitrary behavior bases on what the program tells the interface
>>> to do, you want all of these to be the case. When you are done,
>>> you may want to turn these things back to some previous state
>>> (i.e., the normal IP address, ARP responses per the configured ARP
>>> address of the Ethernet card, and ignoring non-broadcast packets
>>> sent to the interface, since this is the normal behavior of an
>>> ethernet card and reduces system resource consumption (if there
>>> are no promiscuous modes set by other processes.
>> so, we are not providing a way to restore the original behavior,
>> right?
>
> You do another configdev to set things to whatever you want them to be
> when you don't want "raw". The row socket is not persistent or in
> effect past the process, so other processes may do as they please and
> handle other raw or non-raw interfaces - although performance will be
> hit because the interface is looking at everything instead of ignoring
> packets not broadcast or to its MAC address / IP address. The OS keeps
> a list of interfaces and the processes that get each of them at what
> phase in its processing. A raw socket avoids the OS handling (except
> for the low-level driver code to move bytes via the DMA to and from
> the hardware and memory). The configdev code uses IOCTL to alter the
> hardware configuration and tell the OS to put you at an earlier place
> in the list of handlers.
>
>> this should be fixed.
>
> Nothing to fix.

I thought you said that configdev modifies the behavior of both the socket and
the device (e.g., eth0).
I can close the socket and open a new one, but how do I bring the device back
to the original behavior?

>>>> 2. checksum functions: what is their contract? (e.g., min/max
>>>> buffer size? do they modify their argument? how?)
>>>> where specifically are these checksums documented?
>>>> how are these functions supposed to be used?
>>> IP datagrams are limited in total size by RFC791 (http://www.ietf.org/rfc/rfc791.txt
>> looking at page 10, I see Figure 4. (Example Internet Datagram
>> Header).
>> it says that the "Total Length" is in bytes 2 and 3 (0-based, i.e.,
>> bits 16-31)
>> while "Header Checksum" is in bytes 10 and 11.
>>
>> code in ipcsum takes length not buffer[14] and writes the csum into
>> bytes 25&26.
>>
>> I am confused.
>
> That's because the first 14 bytes of an Ethernet frame are (normally)
> for the MAC address. IP datagrams sit inside ethernet frames (or other
> frames depending on the particulars). rawsock assumes an ethernet -
> which is the most common situation. So everything is offset from IP by
> 14 bytes. But ethernet frames are also alterable by rawsock - so you
> can operate many different ethernet MAC addresses. Ethernet frames
> include things like manufacturer identification information and from
> and to MAC addresses (and broadcast addresses), which also have to get
> changed around to send and receive, and have to get changed when
> forwarding if the interfaces deal with other infrastructure elements
> (e.g., switches) that use MAC addresses for decision-making.

I see, so "buffer" is not a "datagram" but a "frame".
what can clisp do with such a frame?
can you give a code sample?

> It gets even worse because loopback (the lo interface) does not have
> ethernet frames (it's not on an ethernet) so for these cases the code
> has to be abused (you do the checksums than shift the bytes to go from
> ethernet to lo, and add your own MAC addresses which have to be right
> to lo datagrams to put then in ethernet frames. rawsock does not do
> this for you.

the csum functions now take :start/:end arguments, so, I guess, you can format
them for loopback too.
actually, I would love to see a piece of code for that.

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: Doing everything with bytes => rawsock

2009-12-01T12:31:57Z

Don Cohen wrote:
> Sam Steingold writes:
> > OS support is on a feature-by-feature basis.
> Which features are supported by which os's and how does one in general
> find out?

either read the sources, or compile and test.

> > tests serve two major goals:
> > 2. Demoing the functionality:
> Of course some of this could be done in the doc.

yes, and I am all for that too.

> > > If you do have root access and a typical network connection, you could
> > > use raw sockets to implement things like ping or traceroute, or
> > > tcpdump but testing these things relies on the behavior of other
> > > machines.
> > ping does not require root.
> Although ping does not require root, an implementation of ping using
> rawsock WOULD require root, because you would be opening a raw socket.

I don't understand this, so I will take your word.
of course, I would much prefer an implementation which would not require root
privileges.
nobody in his right mind would ever run
sudo demo

> - would require a lot of inputs describing things like what network
> interface to use, what IP address, etc. Reasonable defaults for
> these things can generally be determined by various system calls
> or shell commands, but those are also system dependent, even more
> so than raw sockets themselves.

much can be already done, e.g.,

> - the first step would be finding the ip address for www.gnu.org

we already have os:resolve-host-ipaddr.
actually, we have had it for 10.5 years, since 1999-03-10!

> Above only begins to hint at how low level this stuff is. This is a
> problem for demos because there are too many details for anyone to
> want to control, but it's not clear which ones the user (demo viewer)
> will want to control, and each one leads to its own issue of how to
> even find out which options are available. I could write something
> that would work out of MY box, but when it fails on your box you'll
> have to understand a lot in order to figure out why.

this is no good, of course.
can you write a simple general purpose application, like ping or traceroute?

> > > In fact, ideally the tests would make use of a raw socket
> > > interface on that other machine.
> > "ssh host clisp" is your friend.
> So you're now suggesting that the test script require a parameter
> of a partner host directly connected to this one to which you can
> ssh without a password and run clisp on it in a place where the
> necessary code will already be installed?
> This does not sound like a reasonable requirement for a test.
> It might be for a demo, but not a demo that a normal person can
> be expected to run on his own.

how about a trivial ntp client instead then?

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: Doing everything with bytes instead of chars

2009-12-01T11:35:29Z

Sam Steingold writes:

> "configdev sets the IO device"?
This is the sort of thing I'd normally expect to do with a shell
command, or perhaps /proc. And it can be reversed that way also.

> actually, the code only does something when address is non 0.
> (and it only handles ipv4, not ipv6).
Similarly ipv6 addresses can be set by shell command.

> > IP datagrams are limited in total size by RFC791 (http://www.ietf.org/rfc/rfc791.txt
>
> looking at page 10, I see Figure 4. (Example Internet Datagram Header).
> it says that the "Total Length" is in bytes 2 and 3 (0-based, i.e., bits 16-31)
> while "Header Checksum" is in bytes 10 and 11.
>
> code in ipcsum takes length not buffer[14] and writes the csum into bytes 25&26.
looks to me like 24,25

> I am confused.

The buffer starts with the ethernet header I mentioned in some
previous message - 14 bytes.

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: Doing everything with bytes => rawsock

2009-12-01T11:24:34Z

Sam Steingold writes:
> OS support is on a feature-by-feature basis.
Which features are supported by which os's and how does one in general
find out?
> tests serve two major goals:
> 2. Demoing the functionality:
Of course some of this could be done in the doc.

> > If you do have root access and a typical network connection, you could
> > use raw sockets to implement things like ping or traceroute, or
> > tcpdump but testing these things relies on the behavior of other
> > machines.
> ping does not require root.
Although ping does not require root, an implementation of ping using
rawsock WOULD require root, because you would be opening a raw socket.

> if you can write a bare-bones ping/tcpdump/traceroute, I would love
> to add them to rawsock/demos/, similar to new-clx/demos/.
> e.g., can you make it possible to do
> $ clisp -i rawsock/demos/ping -x '(ping:ping "www.gnu.org")'
Yes, it would be pretty straight forward but
- would require root
- would require a lot of inputs describing things like what network
interface to use, what IP address, etc. Reasonable defaults for
these things can generally be determined by various system calls
or shell commands, but those are also system dependent, even more
so than raw sockets themselves.
- the first step would be finding the ip address for www.gnu.org
(unless you're willing to enter the ip address instead)
This is yet another thing that could be done with raw sockets
(dns client), but again requires a lot of system dependent things,
such as what dns server to use -- or where to find out

I suppose the most straight forward and portable way to find the
ip address of www.gnu.org would be to create a tcp socket to it and
read the address from that -- but of course a real ping would be
pretty useless if it required first creating a tcp connection to the
target.

> okay, so the code will not be automatically run.
> it is still very valuable, both as a test (ran by hand) and as a demo.

Above only begins to hint at how low level this stuff is. This is a
problem for demos because there are too many details for anyone to
want to control, but it's not clear which ones the user (demo viewer)
will want to control, and each one leads to its own issue of how to
even find out which options are available. I could write something
that would work out of MY box, but when it fails on your box you'll
have to understand a lot in order to figure out why.

> > In fact, ideally the tests would make use of a raw socket
> > interface on that other machine.
> "ssh host clisp" is your friend.
So you're now suggesting that the test script require a parameter
of a partner host directly connected to this one to which you can
ssh without a password and run clisp on it in a place where the
necessary code will already be installed?
This does not sound like a reasonable requirement for a test.
It might be for a demo, but not a demo that a normal person can
be expected to run on his own.

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: Doing everything with bytes instead of chars

2009-12-01T11:24:32Z

On Dec 1, 2009, at 11:08 AM, Sam Steingold wrote:

> Fred Cohen wrote:
>> configdev sets of the IO device (network card - the "dev" argument)
>> on
>
> "configdev sets the IO device"?

Yes - the interface being used to send and receive. (e.g., eth0, etc.)

>> the identified raw socket (scocket argument) to promiscuous mode
>> (accept all packets and pass them into the program) no ARP mode
>> (the interface does not automatically respond to ARP requests,
>> and configures its built-in address to 0.0.0.0 (which is to say,
>> it has no particular address).
>
> actually, the code only does something when address is non 0.
> (and it only handles ipv4, not ipv6).

Actually, it handles any raw bytes on an interface - the checksum code
is for IPV4

>> Normally, if you want a silent interface which allows true
>> arbitrary behavior bases on what the program tells the interface
>> to do, you want all of these to be the case. When you are done,
>> you may want to turn these things back to some previous state
>> (i.e., the normal IP address, ARP responses per the configured ARP
>> address of the Ethernet card, and ignoring non-broadcast packets
>> sent to the interface, since this is the normal behavior of an
>> ethernet card and reduces system resource consumption (if there
>> are no promiscuous modes set by other processes.
>
> so, we are not providing a way to restore the original behavior,
> right?

You do another configdev to set things to whatever you want them to be
when you don't want "raw". The row socket is not persistent or in
effect past the process, so other processes may do as they please and
handle other raw or non-raw interfaces - although performance will be
hit because the interface is looking at everything instead of ignoring
packets not broadcast or to its MAC address / IP address. The OS keeps
a list of interfaces and the processes that get each of them at what
phase in its processing. A raw socket avoids the OS handling (except
for the low-level driver code to move bytes via the DMA to and from
the hardware and memory). The configdev code uses IOCTL to alter the
hardware configuration and tell the OS to put you at an earlier place
in the list of handlers.

> this should be fixed.

Nothing to fix.

>>> 2. checksum functions: what is their contract? (e.g., min/max
>>> buffer size? do they modify their argument? how?)
>>> where specifically are these checksums documented?
>>> how are these functions supposed to be used?
>> IP datagrams are limited in total size by RFC791 (http://www.ietf.org/rfc/rfc791.txt
>
> looking at page 10, I see Figure 4. (Example Internet Datagram
> Header).
> it says that the "Total Length" is in bytes 2 and 3 (0-based, i.e.,
> bits 16-31)
> while "Header Checksum" is in bytes 10 and 11.
>
> code in ipcsum takes length not buffer[14] and writes the csum into
> bytes 25&26.
>
> I am confused.

That's because the first 14 bytes of an Ethernet frame are (normally)
for the MAC address. IP datagrams sit inside ethernet frames (or other
frames depending on the particulars). rawsock assumes an ethernet -
which is the most common situation. So everything is offset from IP by
14 bytes. But ethernet frames are also alterable by rawsock - so you
can operate many different ethernet MAC addresses. Ethernet frames
include things like manufacturer identification information and from
and to MAC addresses (and broadcast addresses), which also have to get
changed around to send and receive, and have to get changed when
forwarding if the interfaces deal with other infrastructure elements
(e.g., switches) that use MAC addresses for decision-making.

It gets even worse because loopback (the lo interface) does not have
ethernet frames (it's not on an ethernet) so for these cases the code
has to be abused (you do the checksums than shift the bytes to go from
ethernet to lo, and add your own MAC addresses which have to be right
to lo datagrams to put then in ethernet frames. rawsock does not do
this for you.

I hope that helps - but it may merely confuse...

FC
- This communication is confidential to the parties it is intended to
serve -
Fred Cohen & Associates tel/fax: 925-454-0171
http://all.net/ 572 Leona Drive Livermore, CA 94550
Join http://tech.groups.yahoo.com/group/FCA-announce/join for our
mailing list

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: Doing everything with bytes instead of chars

2009-12-01T11:08:53Z

Fred Cohen wrote:
>
> configdev sets of the IO device (network card - the "dev" argument) on

"configdev sets the IO device"?

> the identified raw socket (scocket argument) to promiscuous mode
> (accept all packets and pass them into the program) no ARP mode (the
> interface does not automatically respond to ARP requests, and
> configures its built-in address to 0.0.0.0 (which is to say, it has no
> particular address).

actually, the code only does something when address is non 0.
(and it only handles ipv4, not ipv6).

> Normally, if you want a silent interface which
> allows true arbitrary behavior bases on what the program tells the
> interface to do, you want all of these to be the case. When you are
> done, you may want to turn these things back to some previous state
> (i.e., the normal IP address, ARP responses per the configured ARP
> address of the Ethernet card, and ignoring non-broadcast packets sent
> to the interface, since this is the normal behavior of an ethernet
> card and reduces system resource consumption (if there are no
> promiscuous modes set by other processes.

so, we are not providing a way to restore the original behavior, right?
this should be fixed.

>> 2. checksum functions: what is their contract? (e.g., min/max buffer
>> size? do they modify their argument? how?)
>> where specifically are these checksums documented?
>> how are these functions supposed to be used?
>
> IP datagrams are limited in total size by RFC791 (http://www.ietf.org/rfc/rfc791.txt

looking at page 10, I see Figure 4. (Example Internet Datagram Header).
it says that the "Total Length" is in bytes 2 and 3 (0-based, i.e., bits 16-31)
while "Header Checksum" is in bytes 10 and 11.

code in ipcsum takes length not buffer[14] and writes the csum into bytes 25&26.

I am confused.

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: Doing everything with bytes => rawsock

2009-12-01T10:34:09Z

On Dec 1, 2009, at 10:18 AM, Sam Steingold wrote:

> Don Cohen wrote:
>>
>> It's not clear to me what sort of tests would be appropriate.
>
> e.g., a piece of code which creates a packet, sets the checksums,
> sends it to a
> socket (created, e.g., using socketpair) &c.

The problem is that you cannot send to yourself because the physical
interface that places the bits on the wire does not see them coming
back in. You need a server to do a test.

>> First what OS's are supported?
>
> OS support is on a feature-by-feature basis.
>
> tests serve two major goals:
>
> 1. Testing the functionality. This is not easy to separate from the
> system
> configuration problems (as you point out yourself), including
> portability,
> permissions network availability &c.

The functionality is, unfortunately, too general to do a complete test
- or even close to it.

> 2. Demoing the functionality:
>
>> If you do have root access and a typical network connection, you
>> could
>> use raw sockets to implement things like ping or traceroute, or
>> tcpdump but testing these things relies on the behavior of other
>> machines.
>
> ping does not require root.
> if you can write a bare-bones ping/tcpdump/traceroute, I would love
> to add them
> to rawsock/demos/, similar to new-clx/demos/.
> e.g., can you make it possible to do
>
> $ clisp -i rawsock/demos/ping -x '(ping:ping "www.gnu.org")'

Ping is easy enough, but rawsock lets you do far more and more complex
things. If you want to configure an infrastructure server to support
such tests, we can do it, but be warned, you are unlikely to want what
you are likely to get as a result.

>> Ideally a test would involve another machine directly
>> connected to the one being tested, where that machine is controlled
>> by
>> the test and, of course, is known (or at least assumed) to be working
>> correctly.
>
> okay, so the code will not be automatically run.
> it is still very valuable, both as a test (ran by hand) and as a demo.

I have a variety of tests that I run - but they are part of
proprietary software I provide (also patents apply to parts of the
functionality). The simple thing to do is simply reverse the sending
and receiving IP addresses, checksum the result, and send it back out
the port it came in on. The result will be that whatever they send to
you, you send back. You can then set up 2 processes - one sending
arbitrary content (subject to the proper header format for
transmission across infrastructure), and the other checking the
returns. While mirroring of this sort is patented, I can give you a
free license...

The best approach for a normal user would be to setup a second
machine, both running rawsoc, and send and verify arbitrary byte
sequences pushed out the interface (put them on a LAN by themselves,
generate random sequences, and verify them at the other side. Or you
can do it with one machine that has 2 interfaces... That's what I do.

FC

>> In fact, ideally the tests would make use of a raw socket
>> interface on that other machine.
>
> "ssh host clisp" is your friend.

> ------------------------------------------------------------------------------
> Join us December 9, 2009 for the Red Hat Virtual Experience,
> a free event focused on virtualization and cloud computing.
> Attend in-depth sessions from your desk. Your couch. Anywhere.
> http://p.sf.net/sfu/redhat-sfdev2dev
> _______________________________________________
> clisp-list mailing list
> clisp-list@...
> https://lists.sourceforge.net/lists/listinfo/clisp-list
>
>

From forum: clisp-list

Re: Doing everything with bytes => rawsock

2009-12-01T10:18:22Z

Don Cohen wrote:
>
> It's not clear to me what sort of tests would be appropriate.

e.g., a piece of code which creates a packet, sets the checksums, sends it to a
socket (created, e.g., using socketpair) &c.

> First what OS's are supported?

OS support is on a feature-by-feature basis.

tests serve two major goals:

1. Testing the functionality. This is not easy to separate from the system
configuration problems (as you point out yourself), including portability,
permissions network availability &c.

2. Demoing the functionality:

> If you do have root access and a typical network connection, you could
> use raw sockets to implement things like ping or traceroute, or
> tcpdump but testing these things relies on the behavior of other
> machines.

ping does not require root.
if you can write a bare-bones ping/tcpdump/traceroute, I would love to add them
to rawsock/demos/, similar to new-clx/demos/.
e.g., can you make it possible to do

$ clisp -i rawsock/demos/ping -x '(ping:ping "www.gnu.org")'

> Ideally a test would involve another machine directly
> connected to the one being tested, where that machine is controlled by
> the test and, of course, is known (or at least assumed) to be working
> correctly.

okay, so the code will not be automatically run.
it is still very valuable, both as a test (ran by hand) and as a demo.

> In fact, ideally the tests would make use of a raw socket
> interface on that other machine.

"ssh host clisp" is your friend.

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: Doing everything with bytes => rawsock

2009-12-01T09:54:21Z

Sam Steingold writes:
> great!
> the final touch:
> can you provide a few test cases for clisp/modules/rawsock/test.tst?

It's not clear to me what sort of tests would be appropriate. Other
tests seems to be intended to indicate either that everything is
working or that some particular things are not. In this case the
tests are likely to be partial for several reasons. So perhaps the
notion of a test should be generalized to return another category of
result, a set of things that could not be tested in the current
environment. It occurs to me that the same issues apply also to tcp
sockets, though to a lesser extent.

First what OS's are supported? I expected to see the rawsock section
of impnotes in the platform dependent section. It seems at least as
platform dependent as tcp sockets. I knew it worked in linux and
thought it might work on some other unix versions. I seem to recall
attempts to make it work on mac. I have no idea about windows.

I assume the tests should be run only on supported platforms.
Is that already the case for rawsock? How about tcp sockets?

Second, even on a supported OS you might or might not have the
required OS facilities installed, so tests should be further
conditioned on that. Perhaps you're not too concerned with custom OS
builds, but even without that there are lots of versions and lots of
run time configuration options. Maybe you're happy to view as test
failures whatever problems arise when the environment is missing
facilities used by the test.

Third you might not be connected to a network.

Looking at the current test file I see a lot of #+unix, a fair amount
of localhost (which seems a good idea since that interface seems more
likely than any other to be available), also some comments about root
only. That's another big problem. Raw socket access is generally
available only to root. Even in tcp sockets I think it's common to
restrict listeners on low numbered sockets to root.

If you do have root access and a typical network connection, you could
use raw sockets to implement things like ping or traceroute, or
tcpdump but testing these things relies on the behavior of other
machines. Ideally a test would involve another machine directly
connected to the one being tested, where that machine is controlled by
the test and, of course, is known (or at least assumed) to be working
correctly. In fact, ideally the tests would make use of a raw socket
interface on that other machine.

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: Doing everything with bytes instead of chars

2009-12-01T06:42:51Z

great!
the final touch:
can you provide a few test cases for clisp/modules/rawsock/test.tst?
thanks

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: Doing everything with bytes instead of chars

2009-11-30T16:29:32Z

Sam Steingold writes:
> Don Cohen wrote:
> > ip checksum in rfc 791
The checksum field is the 16 bit one's complement of the one's
complement sum of all 16 bit words in the header. For purposes of
computing the checksum, the value of the checksum field is zero.
bytes 10,11 (starting from zero) of IP header, which can be anywhere
from 20 to 60 bytes long (IHL is second half of byte 0 and is in 4
byte units)

> > udp checksum in rfc 768
bytes 7,8 of udp header which comes right after IP header
Checksum is the 16-bit one's complement of the one's complement sum of a
pseudo header of information from the IP header, the UDP header, and the
data, padded with zero octets at the end (if necessary) to make a
multiple of two octets.

The pseudo header conceptually prefixed to the UDP header contains the
source address, the destination address, the protocol, and the UDP
length. This information gives protection against misrouted datagrams.
This checksum procedure is the same as is used in TCP.

so approximately the entire packet, which could in theory be up to
2^16-1 bytes, but is typically 1500.
> > tcp in rfc 793
same as udp but the tcp header is much longer, and this is bytes 16,17
of that header.

> > icmp in rfc 792
The checksum is the 16-bit ones's complement of the one's
complement sum of the ICMP message starting with the ICMP Type.
For computing the checksum , the checksum field should be zero.
bytes 2,3 of icmp header

> 3. DIUC that every UDP, ICMP & TCP packet is also an IP packet?
> what other inclusions hold?
No packet is more than one of udp,icmp,tcp.
There are non-IP packets and also IP packets other than udp,tcp,icmp.
The "protocol" in byte 9 of an IP packet distinguishes among udp (17 I
think), tcp(6), icmp (1 I think) and other values. The fact that a
packet is IP is encoded in the header before the ip header, I think
it's 0800 - bytes 12,13 in the ethernet header. I don't recall the
rfc for that one. I guess if you use something other than ethernet
it's likely to be somewhere else.

> for each inclusion, specify the correct order in which the checksum
> functions must be called.
I believe that for udp,tcp,icmp packets either order is ok (ip sum
either before or after).
Of course, ipsum should only be used on ip packets, udpsum on udp
packets, etc.

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: Doing everything with bytes instead of chars

2009-11-30T15:55:19Z

Don Cohen wrote:
> ip checksum in rfc 791
> udp checksum in rfc 768
> icmp in rfc 792
> tcp in rfc 793

how about some very basic stuff (I am sure I can fish it from the RFCs, but I
would rather be spoon-fed here).

1. how many bytes do these functions look at. just give me an int.
2. where in the buffer do they write the checksums.
please answer these questions using RFCs only, not the source code.

3. DIUC that every UDP, ICMP & TCP packet is also an IP packet?
what other inclusions hold?
for each inclusion, specify the correct order in which the checksum functions
must be called.

thanks for your replies, they are actually much much more helpful than I ever
hoped for.

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: Doing everything with bytes instead of chars

2009-11-30T15:45:19Z

Fred Cohen writes:
> In particular, you always want to
> do the IPsum after the UDP, ICMP, or TCP checksum so that the IP
> checksum properly checksums the result of the inner checksum
> appropriate to the inner datagram protocol.

I don't think the order matters.
IP csum only covers the IP header.
TCP/UDP csum does not include the IP header - does include some of its
data but not the ip csum.
ICMP csum does not include anything in IP header.

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: Doing everything with bytes instead of chars

2009-11-30T15:24:42Z

On Nov 30, 2009, at 2:25 PM, Sam Steingold wrote:

> Fred Cohen wrote:
>>> ...also, some of your old code could use more comments.
>> I'm not big fan of comments - especially for something that short.
>> If you look at the relevant RFCs, there's very little to it. All
>> it really does is pull in the bytes from the interface, push out
>> bytes to the interface, and perform checksums of various sorts per
>> the definitions in the protocols. The rest of my code is in lisp
>> (and not that well written)... But if you have questions or places
>> you would like a comment, I'd be fine with writing them...
>
> 1. configdev : what it does, how it is supposed to be used.

configdev sets of the IO device (network card - the "dev" argument) on
the identified raw socket (scocket argument) to promiscuous mode
(accept all packets and pass them into the program) no ARP mode (the
interface does not automatically respond to ARP requests, and
configures its built-in address to 0.0.0.0 (which is to say, it has no
particular address). Normally, if you want a silent interface which
allows true arbitrary behavior bases on what the program tells the
interface to do, you want all of these to be the case. When you are
done, you may want to turn these things back to some previous state
(i.e., the normal IP address, ARP responses per the configured ARP
address of the Ethernet card, and ignoring non-broadcast packets sent
to the interface, since this is the normal behavior of an ethernet
card and reduces system resource consumption (if there are no
promiscuous modes set by other processes.

Example:
(configdev socket "eth0" 1 1 ) ; eth0 no arp raw mode on socket

> 2. checksum functions: what is their contract? (e.g., min/max buffer
> size? do they modify their argument? how?)
> where specifically are these checksums documented?
> how are these functions supposed to be used?

IP datagrams are limited in total size by RFC791 (http://www.ietf.org/rfc/rfc791.txt
). They operate on the buffer they are provided with, and alter the
appropriate areas of the datagrams so that the checksum of the
resulting datagram (contained in the buffer they operate on) is
appropriate to the protocol in use. In particular, you always want to
do the IPsum after the UDP, ICMP, or TCP checksum so that the IP
checksum properly checksums the result of the inner checksum
appropriate to the inner datagram protocol.

The IP checksum is documented in RFC791 (http://www.ietf.org/rfc/rfc791.txt
) Example: (ipcsum)
The TCP checksum is documented in RFC793 (http://www.ietf.org/rfc/rfc793.txt
) Example: (tcpcsum) (ipcsum)
The UDP checksum is documented in RFC768 (http://www.ietf.org/rfc/rfc768.txt
) Example: (udpcsum) (ipcsum)
The ICMP checksum is documented in RFC792 (http://www.ietf.org/rfc/rfc792.txt
) Example: (icmpcsum) (ipcsum)

The design is optimized toward receiving a raw packet (which includes
an ethernet header and IP datagram for Internet uses, but may contain
any bit sequence), examining and altering the content of the packet,
and sending it out another interface (as an example). So, for example,
after the sockets are set up, to forward a datagram out another
interface, you simply do a (sendt socket2 (getraw socket1)), which
takes almost no time. On the other hand, if you want to alter the
outbound, for example by replacing the IP address, you simply set the
bytes appropriate for the From IP address to the desired From address,
and do (setf i (getraw socket1)) (ipcsum)(sendt socket2 i).

If, on the other hand, you don;t want to change the checksum, you
don;t have to do so (and may produce infrastructure errors).
Similarly, if you simply want to shove some byte sequence into the
ether(net), you set the bytes into the buffer, and do a sendt

That's about it... any other questions?

FC
- This communication is confidential to the parties it is intended to
serve -
Fred Cohen & Associates tel/fax: 925-454-0171
http://all.net/ 572 Leona Drive Livermore, CA 94550
Join http://tech.groups.yahoo.com/group/FCA-announce/join for our
mailing list

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: Doing everything with bytes instead of chars

2009-11-30T15:20:18Z

Sam Steingold writes:

> 1. configdev : what it does, how it is supposed to be used.

> 2. checksum functions: what is their contract? (e.g., min/max
> buffer size? do they modify their argument? how?) where
> specifically are these checksums documented? how are these
> functions supposed to be used?
ip checksum in rfc 791
udp checksum in rfc 768
icmp in rfc 792
tcp in rfc 793
all say close to the same thing:
The checksum field is the 16 bit one's complement of the one's
complement sum of all 16 bit words in the header. For purposes of
computing the checksum, the value of the checksum field is zero.
They cover different amounts of data, typically headers, see rfc's.
My recollection, which the code seems to confirm, is that they
overwrite the buffer with the correct checksum, which is normally
what you want to do before sending a packet (but, of course, after
any other changes to data you want to send).
If the checksum is wrong on an incoming packet the OS normally
discards it.

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Re: Doing everything with bytes instead of chars

2009-11-30T14:25:14Z

Fred Cohen wrote:

>> ...also, some of your old code could use more comments.
>
> I'm not big fan of comments - especially for something that short. If
> you look at the relevant RFCs, there's very little to it. All it
> really does is pull in the bytes from the interface, push out bytes to
> the interface, and perform checksums of various sorts per the
> definitions in the protocols. The rest of my code is in lisp (and not
> that well written)... But if you have questions or places you would
> like a comment, I'd be fine with writing them...
>

1. configdev : what it does, how it is supposed to be used.

2. checksum functions: what is their contract? (e.g., min/max buffer size? do
they modify their argument? how?)
where specifically are these checksums documented?
how are these functions supposed to be used?

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

clisp-cvs Digest, Vol 43, Issue 19

2009-11-30T04:03:35Z

From forum: clisp-devel

Re: Doing everything at larger scale in lisp

2009-11-28T10:41:44Z

On Nov 28, 2009, at 3:34 PM, Fred Cohen wrote:

> On Nov 28, 2009, at 4:09 AM, Pascal J. Bourguignon wrote:
>
>> ...
>> Search about unicode combining characters for example.
>> http://en.wikipedia.org/wiki/Combining_character
>>
>> A single character may need several unicode code points to be
>> represented. So already, you cannot use a scalar to represent
>> unicode
>> characters in general. Then if you want to store the code points
>> in a
>> vector, that means that the index of the vector doesn't necessarily
>> match the index in the string, so string indexing may not be O(1).
>>
>> Of course, the problem is even more complex when you try to fit the
>> unicode model into the lisp model of characters and strings.
>
> Right - but this has nothing to do with the complexity of characters
> in lisp. There are a virtually unlimited number of different ways to
> encode information.
>
>>> [...] I just want the things contained within the strings to be
>>> bytes (values 0-255) and not altered by any processing except the
>>> specific calls to alter them (no hidden automatic changes please).
>>
>> You need to implement your own type to do that, don't use lisp
>> character and string, they're more sophisticated. As mentionned, you
>> can still have "strings" in source files compiled to your own type of
>> strings, with reader macros.
>
> That's what I was asking about - where can I read about the
> sophistication of strings in lisp?
>
> ...

In the source code of the implementations!

Or if you don't wnat to dig there, just compare CHAR_MAX (in C) vs.
CHAR-CODE-LIMIT (in CL) and infer the consequences.

Compare:

int i=42;
char c=i;

vs.

(let ((i 42)
(c (make-array 1 :element-type 'character :initial-element #\a)))
(setf (aref c 0) i))

and think about what it involves.

Hint: to begin with, character in CL will take 32 bits while char in C
takes 8 bits, therefore any string manipulation will already four time
slower than in C.

Of course when doing I/O it also means converting the internal unicode
codepoints to the external encoding. In C, if you input utf-8 bytes,
you keep them like this (and of course you don't try to take the nth
character of a string, since that'd be O(n)). In Lisp, there will be
decoding and encoding on each I/O.

> All the RAM will not be enough for many of the things I ultimately
> need to do. When you work with multi-terabyte disks, and you are
> trying to do analysis linking things across the entire disk to other
> such things, you need multiple terabytes of memory - which I
> obviously need to do with file systems. I just want a way to not
> have to write everything for file systems, RAM, databases, etc. just
> to get better performance at smaller sizes. Since everybody
> presumably will run into these limits on their programs at some
> point, I was thinking that a virtual RAM that extends to disk
> storage (and ultimately multiple disks across infrastructure) built
> into lisp would be easier (for me) that doing it myself... and for
> everyone who ultimately will have to do it themselves as their
> problems grow in size.

Yes, you will probably be disk bound anyway, so clisp will be as good
as another implementation.

--
__Pascal Bourguignon__
http://www.informatimago.com

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list

Doing everything at larger scale in lisp

2009-11-28T06:34:46Z

On Nov 28, 2009, at 4:09 AM, Pascal J. Bourguignon wrote:

> ...
> Search about unicode combining characters for example.
> http://en.wikipedia.org/wiki/Combining_character
>
> A single character may need several unicode code points to be
> represented. So already, you cannot use a scalar to represent unicode
> characters in general. Then if you want to store the code points in a
> vector, that means that the index of the vector doesn't necessarily
> match the index in the string, so string indexing may not be O(1).
>
> Of course, the problem is even more complex when you try to fit the
> unicode model into the lisp model of characters and strings.

Right - but this has nothing to do with the complexity of characters
in lisp. There are a virtually unlimited number of different ways to
encode information.

>> [...] I just want the things contained within the strings to be
>> bytes (values 0-255) and not altered by any processing except the
>> specific calls to alter them (no hidden automatic changes please).
>
> You need to implement your own type to do that, don't use lisp
> character and string, they're more sophisticated. As mentionned, you
> can still have "strings" in source files compiled to your own type of
> strings, with reader macros.

That's what I was asking about - where can I read about the
sophistication of strings in lisp?

...
>
> Are you aware that there are several implementation of Common Lisp,
> and that clisp is just one implementation of lisp, written in C (hence
> the name, c lisp)?

I am - but I want one that has all of the features of portability, an
outstanding team of people behind it, with a long history, open
source, the ability to interface to C if need be, willingness to talk
to (email with) people like me, in most standard linux distributions,
etc.

> The point here is that, on 32-bit platforms, clisp is rather more
> limited in memory than most other implementations. Check array-total-
> size-limit for example. (On 64-bit platforms, there's enough
> addressing space to use all the available RAM).

All the RAM will not be enough for many of the things I ultimately
need to do. When you work with multi-terabyte disks, and you are
trying to do analysis linking things across the entire disk to other
such things, you need multiple terabytes of memory - which I obviously
need to do with file systems. I just want a way to not have to write
everything for file systems, RAM, databases, etc. just to get better
performance at smaller sizes. Since everybody presumably will run into
these limits on their programs at some point, I was thinking that a
virtual RAM that extends to disk storage (and ultimately multiple
disks across infrastructure) built into lisp would be easier (for me)
that doing it myself... and for everyone who ultimately will have to
do it themselves as their problems grow in size.

> If you need to go to the limit of your addressing space, then sbcl or
> ecl might be better choices than clisp.
>
> Also, since you seem to have needs for performance, while this is more
> difficult to evaluate, it is possible that ecl or sbcl who compile to
> native code (ecl going thru gcc), could produce faster code than clisp
> (on the other hand, clisp mustn't be discarted a-priori, since it has
> now a JIT compiler, and even with a VM, performance can be good for
> some things.

Right - the JIT compiler is extremely helpful since I don't know in
advance what my program may have to do. I also do a substantial number
of things with eval by building up expressions as I go. That's one of
the reasons I prefer an interpretive language - but I still want the
performance of compiled code - which is why clisp fits so well for so
many things.

FC
- This communication is confidential to the parties it is intended to
serve -
Fred Cohen & Associates tel/fax: 925-454-0171
http://all.net/ 572 Leona Drive Livermore, CA 94550
Join http://tech.groups.yahoo.com/group/FCA-announce/join for our
mailing list

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

From forum: clisp-list