summaryrefslogtreecommitdiffstats
path: root/todo
diff options
context:
space:
mode:
authorDaniel Baumann <daniel.baumann@progress-linux.org>2024-04-17 07:42:04 +0000
committerDaniel Baumann <daniel.baumann@progress-linux.org>2024-04-17 07:42:04 +0000
commit0d47952611198ef6b1163f366dc03922d20b1475 (patch)
tree3d840a3b8c0daef0754707bfb9f5e873b6b1ac13 /todo
parentInitial commit. (diff)
downloadnmap-0d47952611198ef6b1163f366dc03922d20b1475.tar.xz
nmap-0d47952611198ef6b1163f366dc03922d20b1475.zip
Adding upstream version 7.94+git20230807.3be01efb1+dfsg.upstream/7.94+git20230807.3be01efb1+dfsgupstream
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'todo')
-rw-r--r--todo/david.txt42
-rw-r--r--todo/djalal.txt146
-rw-r--r--todo/dmiller.txt12
-rw-r--r--todo/done.txt3711
-rw-r--r--todo/gorjan.txt66
-rw-r--r--todo/henri.txt41
-rw-r--r--todo/nmap.txt638
-rw-r--r--todo/nping.txt799
-rw-r--r--todo/patrick.txt77
-rw-r--r--todo/paulino.calderon.txt4
-rw-r--r--todo/sctp.txt49
-rw-r--r--todo/shinnok.txt150
12 files changed, 5735 insertions, 0 deletions
diff --git a/todo/david.txt b/todo/david.txt
new file mode 100644
index 0000000..a9ce685
--- /dev/null
+++ b/todo/david.txt
@@ -0,0 +1,42 @@
+* Make improvements to the irc-unrealircd-backdoor script.
+* Brandon says: "Sometime -sV goes just a little too fast and gets a connect
+ error. It should back off and try again a few times before giving up trying
+ to fingerprint the service." It looks like
+ Got nsock CONNECT response with status ERROR - aborting this service
+ Add a delay of 500 ms?
+Summer of coder:
+* Add a library function to test the randomness of a string. Use it to make
+ version scripts for services that send random or encrypted data, for example
+ cccam on port 12000 which sends 16 bytes.
+
+Zenmap:
+* Do a memory audit of loading a large scan file.
+* Figure out what licensing notices are required in the Mac package for GTK+,
+ Glib, Python, and anything else we use.
+Summer of Coder:
+* Merge a scan aggregation into one XML file.
+* Synthesize text Nmap output from an XML file.
+
+Ncat:
+* Make Ncat send one line at a time when --delay is in effect. This is
+ cumbersome to do until Nsock supports buffered reading.
+* Make the HTTP proxy support the chunked transfer encoding, then change it to
+ be HTTP/1.1 and support pipelining.
+* See if we can make Ncat drop privileges on startup.
+
+Nsock:
+* Add a buffer to each iod, so that you can ask for a certain number of bytes
+ or lines and get exactly that many, no more. Venkat wrote a proposal at
+ http://seclists.org/nmap-dev/2009/q3/0600.html.
+
+Web site:
+* Look for a good online respository viewer.
+
+Done:
+* Handle multiple targets with the same address.
+* Check necessity of mswin32 pcap includes.
+* Try removing the call to PacketSetReadTimeout in readip_pcap, so that Windows
+ uses the short 2 ms timeout like some other platforms without selectable pcap
+ fds do. Measure difference in time and CPU time.
+* Do JavaScript magic to expand/contract NSEDoc sidebar.
+* Check out compression options for the NSIS installer.
diff --git a/todo/djalal.txt b/todo/djalal.txt
new file mode 100644
index 0000000..b674d16
--- /dev/null
+++ b/todo/djalal.txt
@@ -0,0 +1,146 @@
+==
+
+GSoC 2011 TASKS:
+
+o Work on my GSoC vulnerability and exploitation script ideas:
+ https://secwiki.org/w/Nmap/Script_Ideas#Djalal_Harouni
+
+o Review all the "Improve NSE HTTP architecture" proposal suggetions
+ and comments, and try to include them and update the proposal.
+ http://seclists.org/nmap-dev/2011/q2/967
+
+o Start a thread on Nmap-dev about users favorite Nmap and NSE commands,
+ and create a special page for it in the secwiki.org site.
+ This will also let us to create more scan profiles for Zenmap.
+
+==
+
+1) Nmap Scripting Engine Infrastructure:
+
+o [High priority]
+ Take a look at Dan's NSE XML output patch and try to commit it.
+ http://seclists.org/nmap-dev/2011/q2/1230
+
+o NSE Version Numbering.
+ http://seclists.org/nmap-dev/2010/q4/693
+
+[Other tasks]
+o Propose a better duplicate scanned IPs filtering engine.
+
+
+2) NSE Scripts:
+
+[Priorities tasks]
+o NFS/RPC features:
+- add NFS READLINK support to let nfs-ls show symbolic files.
+
+o Review NSE scripts and libs, and fixing bugs:
+ - Document all the new NFS procedures.
+
+[Other tasks]
+o NFS/RPC features:
+- Add more authentication support: Unix authentication.
+- NFSv4 support.
+- Add recursion support to nfs-ls.nse
+
+
+==
+
+MAYBE:
+
+o Create a new rule "versionrule" which will be used by version
+ category scripts.
+ http://seclists.org/nmap-dev/2010/q3/551
+
+o NSE debugger.
+
+o Add more NSE control for long running scripts: one option will be a
+boolean expression filter (like: tcpdump) which will change NSE scripts
+arguments or behaviour according to previous results, this will be
+really useful for big networks. Another option will be a generic NSE
+(Lua) script with an easy and readable code that includes expressions or
+filters selection to let us change NSE arguments according to previous
+results.
+Note: this option will be useful on big networks. however for the moment
+this is a simple idea and it needs further discussion on the nmap-dev.
+
+o Privileges dropping for NSE scripts [nmap TODO list].
+
+o NSE security review [nmap TODO list].
+
+
+o Fixing bugs.
+- NSE not honoring the source port flag when doing version scan.
+ http://seclists.org/nmap-dev/2010/q2/576
+
+ David said that it will not be easy to support setting the source port
+ http://seclists.org/nmap-dev/2010/q3/331
+
+
+==
+
+DONE:
+
+1) Nmap Scripting Engine Infrastructure:
+
+o Submitted the "Improve NSE HTTP architecture" proposal
+ http://seclists.org/nmap-dev/2011/q2/967
+
+o Make NSE scripts able to retrieve the interface network
+ information.
+
+o LuaFileSystem directory iterator [1] port.
+[1] http://keplerproject.github.com/luafilesystem/
+
+o New class of scripts which use two new script rules:
+ - Script Pre-scanning and Script Post-scanning rules: "prerule" and
+ "postrule". Documented these new phases.
+ - Update scripts to use these new rules:
+ dns-zone-transfer now uses "prerule" and "portrule".
+
+o Update other parts of Nmap book to show the new Script scan phases.
+
+o Fixing bugs:
+ - NSE not honoring the Exclude directive bug fixed and committed
+ as r18467.
+
+o Let NSE "prerule", "portrule" and "hostrule" scripts to add new
+discoverd targets to Nmap.
+
+o Update scripting.xml to show the new script scan phases.
+
+
+2) NSE Scripts:
+
+o smtp-vuln-cve2011-1764 script to check Exim DKIM Format String
+ vulnerability (CVE-2011-1764).
+
+o Updated and Improved ftp-vsftpd-backdoor to detect the vsFTPd backdoor
+ (CVE-2011-2523).
+
+o ftp-vuln-cve2010-4221.nse script to check the ProFTPD Telnet IAC stack
+ overflow (CVE-2010-4221).
+
+o smtp-vuln-cve2010-4344 script to check and exploit Exim SMTP Server:
+ heap overflow (CVE-2010-4344) and privileges escalation (CVE-2010-4345)
+
+o SMTP library.
+
+o Rewritten SMTP scripts to use the smtp library:
+ - smtp-commands
+ - smtp-open-relay
+ - smtp-enum-users
+
+o smtp-vuln-cve2011-1720 script to check for CVE-2011-1720
+
+o broadcast-avahi-dos script to check for CVE-2011-1002
+
+o NFS/RPC features:
+ - New script: nfs-ls which combines nfs-dirlist and nfs-acls and try to
+ emulates some features of the old "ls" unix tool. The script support
+ NFSv2 and NFSv3.
+ - Readapted the RPC and NFS library code with a new re-design with new
+ high level functions.
+ - Added NFS procedures support:
+ NFSv2: LOOKUP
+ NFSv3: FSSTAT, FSINFO, READDIRPLUS, PATHCONF, ACCESS, LOOKUP
diff --git a/todo/dmiller.txt b/todo/dmiller.txt
new file mode 100644
index 0000000..0216a3c
--- /dev/null
+++ b/todo/dmiller.txt
@@ -0,0 +1,12 @@
+* Make Zenmap unit tests work. Guessing lots don't, since r32569 fixed real code
+ that matched some unit tests, too.
+
+* Make sure Ndiff, Zenmap are 2to3 compatible with python -3
+
+* Script to check for updated versions of included libs. Have shell for libpcap,
+ but should convert to python.
+
+* NSE stuff
+ * broadcast-srvloc-info - test
+ * broadcast-rpcbind - write, test
+ * Consolidate utility functions
diff --git a/todo/done.txt b/todo/done.txt
new file mode 100644
index 0000000..ccb0a1c
--- /dev/null
+++ b/todo/done.txt
@@ -0,0 +1,3711 @@
+DONE:
+
+o Change Ncat so that it does SSL certificate trust checking by
+ default (even without --ssl-verify) and provides a warning and the key
+ fingerprint if there is no valid trusted chain or the cert is
+ expired, etc. The warning should happen (to STDERR) even if -v is
+ not specified. We should add a new option to force Ncat to quit if
+ cert not valid, and --ssl-verify should become an undocumented alias
+ for that. [GH#30]
+
+o Augment the configure script to list unmet dependencies. Currently, configure
+ works just fine without a C++ compiler installed, but make generates an
+ error. The configure script should be able to detect this. Also, a list of
+ features that are/are-not available would be nice at the end of the script,
+ so folks can see that they've e.g. missed the OpenSSL dependency.
+
+o Add parallel IPv6 reverse DNS support (right now we use the system
+ functions).
+
+o [Ncat] This may sound ridiculous, but I'm starting to think that
+ Ncat should offer a very simple built-in http server (e.g. for simply
+ sharing files, etc.) And maybe a simple client too. (Done via --lua-exec and
+ the httpd.lua script shipped with Ncat)
+
+o INFRASTRUCTURE: Add IPv6 support to secwiki
+ - We probably just have to designate a new IPv6 address for it and
+ add it to Apache config.
+
+o [INFRASTRUCTURE] Improve our main web server http configuration to
+ better handle high load situations and DoS attacks. As part of
+ this, we may have to raise the max client limits. But then there is
+ a risk of running out of RAM, which can be even worse. So we need
+ to figure out a good balance.
+
+o Migrate web.insecure.org to a RHEL-6 derived distro (probably CENTOS
+ 6, since Linode doesn't currently offer ScientificLinux images).
+ o Actually, if we can wait until "second half of 2013", we might be
+ able to jump straight to RHEL 7. And RHEL 5 support looks like it
+ will go on for many more years for critical/security patches.
+ o Maybe start with svn server, since we've had reports of our
+ current one giving people unexpected password prompts. There is a
+ thread about that at http://seclists.org/nmap-dev/2012/q2/17
+ o UPDATE on this - adding read-only rights (rather than no rights)
+ to the root of the svn repo seems to have solved this problem.
+
+o Make Windows 8.1 VM with VS 2013 and do more testing of Nmap compilation/running
+
+o Make and test build on a newer OS X than 10.6 (10.10 was recently released)
+
+o Adopt an issue tracking system for Nmap and related tools. We
+ should probably look at our needs and options and then decide on and
+ either install it on our own infrastructure or use it hosted elsewhere.
+ - David notes that Trac seems to work well for Tor -- see
+ https://trac.torproject.org/projects/tor
+ - One thing which can be nice is being able to interact with the
+ system through email. Like for bugs people file on the Nmap package
+ in Debian, I can just reply to the mail and it gets added in the tracker.
+ - This is now live at http://issues.nmap.org/
+
+o Update OpenSSL library to 1.0.1j
+
+o Our "make uninstall" should uninstall ndiff if it was installed too.
+  We should probably do it in pretty much the same way we handle
+  Zenmap (configure.ac, Makefile.in, and ndiff/setup.pl)
+
+o Web: We should probably distribute RapidSSL intermediate certificate
+ on SecWiki so it is trusted even if browsers don't have that cert
+ cached. Here's a page nothing the issue:
+ https://www.ssllabs.com/ssltest/analyze.html?d=secwiki.org
+ - We probably need to add an entry in apache conf after
+ SSLCertificateFile which looks something like:
+ SSLCertificateChainFile /etc/apache2/rapidssl.pem
+
+o The XML version of Nmap lists and describes the six port states
+ recognized by Nmap near the top of the "Port Scanning Basics"
+ section.  That can be seen in the HTML rendering at
+ https://nmap.org/book/man-port-scanning-basics.html.  But in the man
+ page (nroff) rendering, the list is missing and it just gives the
+ title: "The six port states recognized by Nmap".  UPDATE: Now the
+ descriptions for each state appear in the man page, but the headings
+ ("open", etc.) are missing. We should figure out
+ why, and fix it.
+ - The bug in the stylesheets means that (From Daniel): "if you have an <indexterm>
+ element and it's followed by anything other than whitespace+CDATA
+ (like "</indexterm> foo") then the remaining cdata or element until
+ the next new element will be nroff-commented so this
+ <indexterm>blah</indexterm> is ok, but this <indexterm>blah</indexterm>, is not ok because of the commaand this <indexterm>blah</indexterm> <command>nmap -A</command> is bad no matter how much whitespace intervenes"
+
+
+o Fix a segmentation fault in Ncat when scanned with the SSL NSE
+ scripts. I was able to reproduce this on 2013-09-27 with latest SVN
+ by running:
+ Ncat: ncat -v -k --ssl -l localhost
+ Nmap: ./nmap --script-trace --script '+ssl*' localhost -p 31337
+ This was initially reported by Timo Juhani Lindfors on the Debian
+ bug tracker: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=724580
+ Henri notes: "I traced the latter back to openssl and opened a
+ ticket there, which never got any reply... https://rt.openssl.org/Ticket/Display.html?id=2885&user=guest&pass=guest"
+
+o Investigate how we're ending up with OS fingerprints in nmap-os-db
+ with attribute names like W0 and W8 when according to the docs they
+ are only supposed to be W1 - W6 (and plain W).
+ https://nmap.org/book/osdetect-methods.html#osdetect-w. See also
+ http://seclists.org/nmap-dev/2013/q4/68. Need to determine how
+ these are getting into the file (from Nmap itself or our
+ integration/merge tools) and fix that then remove them from the
+ file.
+
+o Integrate latest IPv4 OS detection submissions and corrections
+
+o We should improve the Windows build process for Ndiff, since it
+ works differently now that it is modularized. To build the Nmap
+ 6.45 release, we (as a temporary hack, not in SVN):
+ - Added 'ndiff' to zenmap/setup.py 'packages' list in
+ COMMON_SETUP_ARGS
+ - Created a zenmap/ndiff subdir (empty) and copy ndiff/ndiff.py into zenmap/ before build.
+ We should find a more elegant solution and check it into SVN. The
+ fundamental issue is that the ndiff.exe we generate needs to be
+ able to access the new ndiff.py module.
+ Also, we need to make sure the -win32.zip Nmap distribution works
+ properly.
+
+o [Zenmap] Combine parallel timed-out hops into one node in the
+ topology view. http://seclists.org/nmap-dev/2012/q1/82 has a patch,
+ however it doesn't handle the case of two or more consecutive
+ timeouts.
+
+o If Nmap uses a "tcpwrapped" port to do fingerprinting on, OS detection
+ might give false matches/results. Since it doesn't really matter which
+ open port gets chosen, we should move onto another open port if we
+ notice "tcpwrapped".
+
+o Implement an --exclude-ports option. See
+ http://seclists.org/nmap-dev/2012/q1/275
+
+o In an ideal world, Zenmap would not run out of memory and crash.
+ And we already have an entry for improving Zenmap's memory
+ consumption. But in the meantime, we should catch the error and
+ present a more useful error message/explanation so the user
+ understands the problem. This should reduce the number of
+ out-of-memory "crash reports" we get too. See
+ http://seclists.org/nmap-dev/2014/q2/298
+
+o Provide an option to send a comment in scan packet data for target
+ network. Examples: --data-string "Scan conducted by Marc Reis from
+ SecOps, extension 2147" or --data-string "pH33r my l3eT
+ s|&lt;iLLz! I'll 0wN UR b0x!"
+
+o We should probably update our included libpcap. We currently
+ include version 1.2.1 (we upgraded to that in April 2012) while the
+ latest version on tcpdump.org is 1.5.3. We make minor changes to
+ libpcap that we ship, and instructions for upgrading are in
+ libpcap/NMAP_MODIFICATIONS.
+
+o Investigate report of Nmap ARP discovery using the wrong target MAC
+ address field in ARP requests (it is correct in the ethernet frame
+ itself). See this thread: http://seclists.org/nmap-dev/2011/q3/547
+
+o Add randomizer to configure script so that a random ASCII art from
+ docs/leet-nmap-ascii-art*.txt is printed. I think I'll start naming
+ them leet-nmap-ascii-art-submittername.txt.
+
+o Add IPv6 subnet/pattern support like we offer for IPv4.
+ o OK, we now have the subnet/pattern support, but not the two-stage
+ model discussed below. So we added a separate task for that.
+ o Obviously we can't go scanning a /48 in IPv6, but small subnets do
+ make sense in some cases. For example, the VPS hosting company
+ Linode assigns only one IPv6 address per user (unless they pay)
+ and you can find many Linode machines by scanning certain /112's.
+ And patterns might be useful because people assigned /64's might
+ still put their machines at ::1, ::2, etc.
+ o David says: "We need to design a new way to iterate over host
+ specifications (i.e., different than nexthost). Because the new
+ host discovery code is sometimes going to want whole netblocks
+ and sometimes individual hosts. So I'm thinking of a two-stage
+ model, where the iterator will received (parsed) specifications
+ like AAAA::1/48, and then it can decide whether to further
+ iterate that into individual addresses, or pass the block off
+ to some specialized discovery routine."
+
+
+o Consider implementing RPC scan with ultra_scan or something else.
+ Right now it is the only program using pos_scan. On the other hand,
+ I'm not sure TCP RPC scanning is appropriate for ultra_scan.
+
+o When Ncat is compiled without OpenSSL, we should still accept the
+ --ssl argument and just give an error message noting that SSL was not
+ compiled in. This reduces confusion for users
+ (e.g. http://seclists.org/nmap-dev/2013/q3/579)
+
+o We should update our OpenSSL Windows binaries from version 1.0.1c to
+ something newer, like 1.01f
+
+o Web: figure out why autogeneration of nmap.org/nsedoc/ doesn't seem
+ to be working. I think we had a cron job which was supposed to be
+ doing it.
+ - hb system was still running crontab files from old web vm in its
+ rc.local. Fixed.
+
+o Add a W3C XML Schema Definition (XSD) for Nmap XML output. Keeping the DTD
+ around is also helpful, but XSD is widely supported and could help improve
+ support for Nmap XML in other tools.
+ o We're going to discuss this on mailing list before deciding
+ whether to 1) switch from DTD to XSD, 2) stick with just a DTD, or
+ 3) try to support both.
+
+o Update copyright year to 2013 in the Nmap copyright header files
+
+o Update CHANGELOG for new release
+
+o New Nmap Release
+
+o Nping in ICMP mode (default) must not be checking the icmp IDs or
+ returned packets or something, because if I have two separate 'nping
+ scanme.nmap.org' running at the same time, each nping sees the replies
+ from the other nping (as well as its own) and it screws up the timing
+ stats too.
+
+o Process Nmap OS service detection submissions
+ - New fingerprints + corrections
+ - Last done November 2012: http://seclists.org/nmap-dev/2012/q4/222
+
+o Process Nmap IPv6 OS detection submissions
+ - New fingerprints + corrections
+
+o Process Nmap IPv4 OS detection submissions
+ - New fingerprints + corrections
+ - Last done in November 2012: http://seclists.org/nmap-dev/2012/q4/221
+
+o Make Ncat reset the signal handler for SIGPIPE to SIG_DFL before
+ execing a program with --exec and friends. A "broken pipe" error in
+ a subprocess should kill the subprocess. Lack of default SIGPIPE
+ handling is what prevents a trivial Lua chargen script--it loops
+ forever after the socket disconnects because none of its writes
+ fail. Cf. http://www.chiark.greenend.org.uk/ucgi/~cjwatson/blosxom/2009-07-02-python-sigpipe.html.
+
+o [Nping] In '-q' mode, Nping should keep the line giving the min/max/avg rtt
+ times. That way people can avoid seeing each individual packet but
+ still see the stats which are similar to what normal ping gives
+ them.
+
+o [Nping] Remove the lines starting with 'Tx time' and 'Rx time' by
+ default (and of course quieter modes), but leave them for cases at
+ least one level of -v.
+
+o Nping/Nmap should probably show ICMP ping sequence values by default
+ in packet trace mode. This would be nice for Nping since that is
+ the default ping it sends and is the main way to distinguish the
+ packets since the IPIDs are the same.
+
+o Complete migration away from Syn colocated machine
+ - [Done - actually was already on web] Move submission CGIs to web
+ - Make sure notification still works
+ - [Done] Mailman
+ - [Done] Install mailman software on web, including CGIs
+ - Migrate mailing lists to web
+
+o Remove the -q/FAKE_ARGV stuff from Nmap, since I don't think people
+ use that any more.
+
+o We should document Ron's sample script
+ (https://nmap.org/svn/docs/sample-script.nse) in docs/scripting.xml
+ so that new script writers know about it.
+ - Decided to remove it instead. Justification: "It is a great idea,
+ but nobody seems to use it (for example, there were no replies to
+ usage inquiry here: http://seclists.org/nmap-dev/2012/q4/379). I
+ think there are two main uses for this script, both of which are
+ being served by other resources. 1) as a template for new
+ scripts. Users instead seem to pick a script that is most similar
+ to the one they want to write and start with that. 2) As a way to
+ learn more about the format of an NSE script. Users instead seem
+ to use our documentation
+ (https://nmap.org/book/nse-script-format.html). So I'm deleting it
+ for now. But if folks miss it, they're welcome and encouraged to
+ say so on dev@nmap.org and we could consider putting it back
+ and/or improving it"
+
+o Upgrade Mac Mini to Mac OS X 10.8 (Mountain Lion) and test building
+ as well as testing usage of our normal builds (which we currently
+ build on 10.6).
+
+o Make a branch from the 6.20BETA1 release (r30266) for new stable
+ release, apply any important bugfix patches from the meantime and then
+ release it after Thanksgiving as new Stable release.
+
+o [NSE] We may want to consider a better exception handling method --
+ one which doesn't require wrapping every I/O line in its own try
+ function call. David says "Lua has an internal "exception handling"
+ mechanism based on a function called pcall, which is implemented
+ with setjmp/longjmp. You can wrap a function call in it and the
+ function will return there whenever there's an unhandled error.
+ Something based on that would be better [than the current system], I
+ think."
+ - This one is obsolete as the Lua 5.2 now lets you do a Lua yield
+ across C function calls.
+
+o Add IPv6 support to Nping, including raw packet mode (hopefully
+ sharing as much code with Nmap as possible, though Nping's packet code
+ is a bit different), and also including echo mode server and client
+ support.
+
+o Make sure we update everywhere relevant (e.g. refguide, etc.) to
+ note the addition in Nmap of the Liblinear library for large linear
+ classification (http://www.csie.ntu.edu.tw/~cjlin/liblinear/). It
+ uses a three-clause BSD license:
+ http://www.csie.ntu.edu.tw/~cjlin/liblinear/COPYRIGHT
+ - David has added it to 3rd-party-licenses.txt
+ - Fyodor moved it into the refguide
+
+o Consider including OpenSSL in our Nmap tarball
+ - Need to check the size, etc.
+ - OK, we're counting this as done because we took all the Win
+ binaries out of the tarball and put them in an nmap-mswin32-aux svn
+ directory which users check out to compile Nmap on Windows, and
+ OpenSSL is included in this.
+
+o Update the Nmap CHANGELOG for latest improvements
+
+o Do an Nmap dev release. Last release was Nmap 6.01 June 22.
+ o Update Nmap version number and auto-generated files for release.
+
+o Process latest Nmap OS submissions and corrections (IPv4 and IPv6).
+ Last done (for IPv4 anyway) in February 2012.
+
+o Review and consider integrating Tomas Hozza's UNIX-domain socket
+ support patch for nsock/ncat: http://seclists.org/nmap-dev/2012/q4/24.
+
+o Improve CPE coverage in OS detection DB from 84% to 90% (see CPE
+ entry a ways down for more on this).
+
+o Process latest service detection submissions. They were last done
+ in February 2012.
+
+o Integrate Henri's new kqueue/poll nsock-engines support.
+
+o If it is trivial to add, it would be nice if the "New VA Module
+ Alert Service" also gave the Author field for NSE scripts so everyone
+ knows which hero(es) wrote it.
+
+o Clean up the Nmap repo to remove some bloat we've allowed to creep
+ in. Should do a more thorough search, but for now here are two
+ obvious candidates:
+ - Create publicly readable /nmap-mswin32-aux in svn
+ - Files not needed for compiling Nmap itself (e.g. only needed for
+ creating or including in Nmap packages), particularly including the
+ vcredist files, should be moved to new /nmap-mswin32-aux
+ - The /nmap-mswin32-aux files won't be included in Nmap tarballs
+ either
+ - Add the gtk, glib, etc. Windows dependencies to /nmap-mswin32-aux
+ so users don't have to all install those in order to compile Zenmap
+ and make Nmap packages.
+ - move the nmap-private-dev/mswin32 stuff into /nmap-mswin32-aux
+ - Update nmap-install.xml for new changes. Such as noting need to
+ checkout this new directory for building packages, removing the
+ need to install your own gtk, glib, etc.
+ - [done] Remove the 5MB of XSL in nping/docs/xsl
+
+o Update our mswin32/OpenSSL to newest version (previous update was
+ September 2010 to 1.0.0a).
+
+o Nmap should have a better way to handle XML script output.
+ o done: https://nmap.org/book/nse-api.html#nse-structured-output
+ o We currently just stick the current script output text into an XML tag.
+ o Daniel Miller is working on an implementation:
+ https://secwiki.org/w/Nmap/Structured_Script_Output
+
+o Update more web content in real time (or near real-time, or at least
+ on an automated basis rather than requiring manual checkin and
+ update). In particular:
+ o NSEDoc generation
+ o [done] SVN dir (https://nmap.org/svn/) should be removed and a redirect
+ added to https svn server.
+ o Maybe Nmap book building
+ o Maybe the generated files in nmap.org/data/
+
+o Update web.insecure.org so that rather than requiring us to build
+ nsedoc on other machines, check it into svn, and then update svn on
+ web, it is done by a script on web which could be run through cron
+ (and potentially from a simple svn commit hook) to build them on the
+ web server directly.
+ - There are other similar things we might want to automate later,
+ such as book rebuilding when the XML files are changed.
+
+o Investigate/fix potential routing-related issue. See emails from
+ Djalal and others: http://seclists.org/nmap-dev/2012/q3/116,
+ http://seclists.org/nmap-dev/2012/q3/4,
+ http://seclists.org/nmap-dev/2012/q2/449
+
+o Even without the --osscan-guess flag, Nmap should show the closest
+ matches (if they pass our threshold) in the XML output. We omit
+ them from the normal output in large part to encourage people to
+ submit fingerprints, but that argument doesn't apply so well to XML
+ output users. Normal output users who really want to see the Nmap
+ guesses could still use --osscan-guess as before.
+
+o Change the interface of nmap.ip_send to take an explicit
+ destination address. It currently extracts the destination from
+ the packet buffer, which does not have enough information to
+ reconstruct link-local addresses. See r26621 for a similar change
+ that was made to Nmap internals.
+
+o [Zenmap] Install higher-resolution icons (at least 64x64 and maybe
+ up to 512x512). Here is a screenshot of the current 48x48 icon on
+ GNOME 3: http://seclists.org/nmap-dev/2012/q2/395.
+ o Sean did Windows and Linux icons, and David did the Mac
+ one.
+
+
+o [NPING] At least on my (Fyodor) system, I get errors like "READ-PCAP
+ killed: Resource temporarily unavailable" with some commands.
+ Example:
+ # nping --tcp -p80 -c1 scanme.nmap.org
+
+ Starting Nping 0.5.61TEST4 ( https://nmap.org/nping ) at 2012-02-16 17:52 PST
+ SENT (0.3307s) TCP 192.168.0.5:42005 > 74.207.244.221:80 S ttl=64 id=23109 iplen=40 seq=1015357225 win=1480
+ RCVD (0.3524s) TCP 74.207.244.221:80 > 192.168.0.5:42005 SA ttl=51 id=0 iplen=44 seq=3197025741 win=14600 <mss 1460>
+ nping_event_handler(): READ-PCAP killed: Resource temporarily unavailable
+ nping_event_handler(): TIMER killed: Resource temporarily unavailable
+ [...]
+
+o [NPING] Nping should probably give you an error or warning when you
+ do: "nping -p80 google.com" since it is ignoring the port specifier.
+ The user probably wants to add --tcp.
+
+o Investigate why http pipelining so often doesn't work in NSE
+ scripts, and often NSE ends up reverting to one request at a time.
+ Scripts may not be using it correctly, and also we wish it were more
+ transparent and there wasn't this big API divide between pipeline
+ and non-pipeline. We just want it send requests as fast as it can,
+ and get a callback when there's a response. Maybe the http library
+ buffers them, or pipelines them, or blocks the http.get call until
+ there's more room. It just seems to always degenerate to 1 request
+ at a time. For example:
+ sudo nmap --script=http-enum bamsoftware.com -p80 -d2
+ quickly (within a few seconds) gives:
+ NSE: http-enum: Searching for entries under path '' (change with 'http-enum.basepath' argument)
+ NSE: Total number of pipelined requests: 2081
+ NSE: Number of requests allowed by pipeline: 100
+ NSE: Received only 41 of 100 expected responses.
+ Decreasing max pipelined requests to 41.
+ NSE: Received only 1 of 41 expected responses.
+ Decreasing max pipelined requests to 1.
+ 100 may a wildly high number of requests to attempt to pipeline.
+ And then something else probably goes wrong after it decides 41 is okay.
+ - Related: Does caching work with pipeleined requests? We should
+ make sure it does.
+ [ OK, the main part of this todo item is done. Though there is a
+ patch pending from Piotr which changes how pipelining works that
+ is worth considering. We did fix the underlying pipelining bug, but
+ (just as with most browsers), it isn't enabled by default. Also, it
+ doesn't support caching. See
+ http://seclists.org/nmap-dev/2012/q3/616. ]
+
+o Make Nmap from a clean start (e.g. after make clean or whatever, so
+ it compiles everything) and research all the compile warnings to see
+ which ones can be fixed/removed. Of course caution is needed to
+ make sure we don't cause problems. For example, an unused variable
+ on one platform might not be unused on another, so we can't just
+ remove it. May have to surround it by ifdefs though.
+
+o Solve "spurious closed port detection" issue discovered by David:
+ http://seclists.org/nmap-dev/2012/q1/62 . So we need to figure out
+ what is going on here and then how to fix it. Note that this
+ doesn't seem to happen when you do ICMP host discovery first (-PE),
+ so it probably relates to the ACK packet that Nmap sends to port 80
+ on the target by default.
+
+o Add real headers for more protocol types in -6 -sO scan. Dario
+ Ciccarone provided some packet captures for
+ 0x00: hop-by-hop
+ 0x2b: routing
+ 0x2c: fragment
+ 0x3c: destination
+ (http://seclists.org/nmap-dev/2011/q2/1003). We also have examples
+ of crafting some of these in FPEngine.cc. [Sean and David]
+
+
+o Investigate increasing FD_SETSIZE on Windows to allow us to
+ multiplex more sockets. See Henri's email:
+ http://seclists.org/nmap-dev/2012/q1/267
+ [James Rogers did some investigative work on this in July 2012, but
+ we weren't able to find a great solution. Maybe we should
+ investigate this more in the future, and also investigate other
+ Windows socket APIs such as completion ports. ]
+
+o Finish sv-tidy - a program to canonicalize and tidy nmap-service-probes.
+ o Check for the same reference (like $1) being used in unrelated fields
+ (where related fields are the pairs (p, cpe:), (v, cpe:), (i, cpe:),
+ (o, cpe:)).
+ For example if we have v/$1/ h/$1/ it is a bug.
+ o Check a list of common product names that should only appear in p//,
+ not in i//. We still have entries that are like this:
+ p/Foobar 2000 ADSL router/ i/micro_httpd web server/
+ that should rather be written this way:
+ p/micro_httpd/ i/Foobar 2000 ADSL router/
+ o [Done] Check for e.g. i/French/ without :fr in cpe:/a, and vice versa.
+ [Sean and David?]
+
+o Remove Nmap's --log-errors feature and make its behavior the
+ default. A few notes:
+ - Nmap should just ignore --log-errors if it sees it
+ - Remember to remove it from the documentation
+
+o We should probably sort script output (for port output and host
+ output) by script name or something so that it comes in a
+ deterministic order. If the same three scripts produce output in
+ two different scans, they should be listed in the same order. Right
+ now the order can vary, at least for host output.
+ [Sean]
+
+o Add a function such as --disable-arp-ping which prevents hosts from
+ being automatically detected as 'up' just because they responded to
+ ARP. Instead, Nmap will actually send the requested host discovery
+ probes (ICMP ping packets, SYN packets, etc.) and only mark the host
+ as up if it responds on an IP level. This is how machines are
+ already treated if they're not on the local network (e.g. if ARP
+ discovery is unavailable). This technique is a bit slower and more
+ likely to miss hosts (e.g. if they're heavily firewalled) than ARP
+ discovery, but the option is needed to handle local networks which use
+ proxy ARP, which would otherwise cause all IPs to appear to be up.
+
+o We should add fields to the service submitter [James is working on this]
+ (http://insecure.org/cgi-bin/submit.cgi?new-service) for the
+ application name and version.
+ o We also need to ensure all fields of /cgi-bin/submit.cgi have
+ proper escapting to prevent possible reflected XSS attacks
+ reported by Maxim Rupp (@mmrupp). The risk is low, if any, since
+ we don't give authentication cookies for bad guys to steal, but is
+ still better to properly escape.
+ o If we get a chance, would be interesting to run our XSS-testing
+ NSE scripts against this and see if they locate the problems.
+ o Also, need to change the font family in there from "Lucida Grand"
+ to "Lucida Grande"? Just a typo. And fix "WIkipedai". We should
+ just spell-check all the output
+
+o Make Nmap 6.01 release containing (among possibly other little
+fixes)
+ - Python upgrade
+ - [done] Zenmap 10.7 hang fix (done in trunk)
+ - [done] Zenmap crash when filtering hosts (done in trunk)
+ - [done] get_srcaddr fix (done in trunk)
+
+o Upgrade Python on build machines to try and resolve Python 2.7
+ security warning (it doesn't affect us, but can worry users). See
+ this thread: http://seclists.org/nmap-dev/2012/q2/621
+
+o Fix get_srcaddr error happening on Windows XP
+
+o [Web] Add a page with the Nmap related videos we do have already
+ - We have a page on Secwiki now: https://secwiki.org/w/Nmap/Presentations
+
+o Zenmap hang on OS X 10.7
+
+o For many years, the Nmap man page and online documentation has had
+ an "Inappropriate Usage" section which notes that "Nmap should never
+ be installed with special privileges (e.g. suid root) for security
+ reasons". And of course Nmap's official installer would never
+ install Nmap that way. While one would thinks that would be enough,
+ we might want to go even further and have Nmap detect when it is run
+ suid and print a security warning.
+
+o Prepare release notes, web page, etc.
+
+o Do private beta release
+
+o Make the release
+
+o In Nmap XML output, osclass (OS Classification) tags should be
+ children of osmatch (the human readable OS name line) rather than
+ having Nmap deduplicate all the osclasses and put them in as
+ siblings. But this change might break some systems which utilize
+ Nmap XML output, so, along with this change, we need to introduce an
+ option such as --deprecated-osclass-xml to return the old behavior.
+ That option only needs to be documented in the CHANGELOG entry
+ referring to this change, and it should note that we're likely to
+ remove this option in a year or two.
+
+o Right now, when an IPv4 or IPv6 address seems bogus (such as 1.2.3
+ or 2001::0 in IPv4 mode), we give a fatal error and abort the scan.
+ But since that might just be one bad target in a long list of hosts to
+ be scanned, it is probably better to just print a warning and
+ continue. Some sort of warning or host element should be included in
+ the XML to explain what happened too. This should also happen if
+ we're unable to resolve a DNS name.
+
+o In sv-tidy, check that used references start at 1 and are
+ contiguous. If $1 and $3 are used but not $2, it's probably a bug.
+ Maybe you can even find out how many there should be by inspecting
+ the regular expression.
+
+o Raw scans from Mac OS X seems not to retrieve the MAC address or do
+ ARP ping, except when scanning the router on an interface. For
+ example, scanning 192.168.0.1-5 sends ARP pings to 192.168.0.1, but
+ the normal four-probe combination to the other addresses. The "MAC
+ address:" line appears in the output for .1 but not for the others.
+
+o To avoid Nmap memory usage bloat, find a way for NSE scripts to
+ store information about a host which expires after Nmap is done
+ scanning that host (e.g. when the hostgroup containing that host is
+ finished). Right now scripts store such information in the registry
+ and it persists forever. For example, a web spidering
+ script/library could store information about the web structure and
+ even page contents so that other scripts can use that information
+ without spidering the target again, but ensuring that the memory
+ will be freed after the hostgroup finishes so there is room to store
+ the web information for the next group of systems. One idea would
+ be to make a host.registry member which contains a registry specific
+ to a specific target. Scripts could store temporary information
+ there, but still use the global registry for information which must
+ persist (e.g. to be used by postrules, etc.)
+
+o Add CPE support to IPv6 OS detection
+
+o Use BPF libpcap logic on Solaris 11, otherwise packet capture doesn't
+ work at all. http://seclists.org/nmap-dev/2012/q1/613
+
+o [NSE] host.os should not just be a list of strings which can contain
+ human-readible strings and/or CPE info. It should probably be list
+ of host.os tables which can contain:
+ host.os[].name <-- human readible name
+ host.os[].class[].vendor
+ host.os[].class[].osfamily
+ host.os[].class[].osgen
+ host.os[].class[].devicetype
+ host.os[].class[].cpe[] <-- array of cpe:/ strings
+ So host.os[1].class[1].cpe[1] is the first CPE entry for the first
+ classification of the first OS match for the target system.
+ The host.os entry docs/scripting.xml would have to be updated too.
+
+o We should probably go through the nmap-os-db (and IPv6 version)
+ entries and, where the fingerprint line specifies a service pack
+ number (or even two of them), ensure that we have sp-qualified CPE
+ entries like "cpe:/o:microsoft:windows_xp::sp2". Right now we
+ sometimes include the qualification, and sometimes not.
+ o This is best done with cpeify-os.py, if possible.
+
+o Zenmap no longer ads the installed module directory to its module
+ search path because some distributors first install in a world
+ writeable directory (like /tmp) and then put those files into their
+ packages which they distribute to users. But this change can lead
+ to Zenmap not working for users who install in nonsystem areas like
+ their home directory (e.g. --prefix /home/fyodor) unless they have
+ their PYTHONPATH set to find them. We should implement a solution,
+ such as making sure Zenmap catches the missing modules error and
+ suggest that the user set their PYTHONPATH or something.
+
+o Scans from Mac OS X tend to use raw IP packets rather than ethernet
+ frames even on the local network because Dnet does not seem to be
+ retrieving the routing table properly -- so the LAN doesn't even
+ show up in --iflist. Patrik can reproduce this on all 3 of his
+ MACs (OS X versions 10.7.3). Comparing the code in DNet route-bsd.c
+ to Apple's own routing table code discovered by Patrik suggests that
+ the Dnet code may be incorrect.
+
+o ssl-google-cert-catalog should not require that the user specify
+ ssl-cert in order to run. Instead, they should probably both call a
+ library which obtains the certificate (and caches it so that it
+ doesn't happen twice if both scripts are run). In general, we want
+ to avoid having any scripts tell the user "this script only works if
+ you specify this other script too". If we really find we need that
+ functionality, we should add a "strong dependencies" feature so that
+ scripts can tell Nmap what other scripts they require.
+ [Patrik did this by adding an ssl cert library]
+
+o Our targets-ipv6-multicast-slaac.nse should probably send the router
+ advertisements with low priority to reduce the chances of any
+ negative impacts on clients, if we're not doing that already. See
+ http://lists.si6networks.com/pipermail/ipv6hackers/2012-March/000503.html.
+ - Actually, I think we already do this. Marking as done.
+
+o Deal with the issue of timeouts happening too soon due to global
+ congestion control in some cases. For example, if Nmap sends host
+ discovery probes to two hosts, and one comes back extremely quickly,
+ it can cause the global congestion control to use a very low timeout
+ and cause the 2nd host (which doesn't have any host-based congestion
+ control values yet) to timeout arguably too quickly. We should look
+ at potential algorithm changes to improve this.
+ David: I think I was wrong about the cause of this. Even when
+ replies come back very quickly, the timeout is by default limited
+ to 100000 microseconds, much higher than the straightforward
+ calculation would give. What I think is really happening is that
+ select is not working reliably on this platform (Solaris 10 x86).
+ In the loop in read_arp_reply_pcap, pcap_select returns 1, then a
+ pcap_next is done. Then pcap_select returns 0, but if I insert
+ another pcap_next after that, the pcap_next finds another packet
+ without blocking (the first time, anyway; after that it blocks).
+
+o Create CHANGELOG
+
+o Make stable release candidate branch
+
+o Make at least one more test release from the candidate branch
+
+o Write and send GSoC 2011 results email
+
+o Document the nsearg format changes made by Paulino (how you can
+ preface an argument with a script to make it more specific, or make it
+ general to apply to multiple scripts)
+ o Rough drafts:
+ o nmap-exp/calderon/refguide.xml
+ o nmap-exp/calderon/scripting.xml
+ o Relates to:
+ o We should probably modify stdnse.get_script_args so that it first
+ checks [scriptname].[argname] and then (if that fails) looks for
+ [argname] by itself. This way people who are only running one
+ script or who want to use the same value for multiple scripts that
+ take the same argument can just give [argname]. But those who want
+ an argument to only apply to a specific script can give
+ [scriptname].[argname].
+
+o Make the nmap.header.tmpl wording a little more generic so it more
+ clearly applies to Ncat, Zenmap, Nping, etc. Then use
+ templatereplace.pl to apply those changes to the code. [Fyodor]
+
+o Change Nmap copyright dates (in the file headers, etc.) from 2011 to
+ 2012.
+
+o Get RPM staticly linking to libsvn (rather than dynamic linking) so
+ that it isn't a requirement for installing the RPM.
+ - We decided to just make nmap-update its own separate RPM so that
+ it can dynamically link to libsvn without forcing that dependency on
+ the whole nmap RPM package.
+ - since the libsvn-devel package apparently only installs dynamic
+ libs, we'll probably have to install it ourselves on the CentOS
+ build machines.
+
+o Fix "BOGUS! Can't parse supposed IP packet" in packet trace of IPv6
+ packets.
+
+o Integrate latest IPv6 OS detection fingerprint submissions
+ - In addition to the submission CGI submissions, some were emailed to Fyodor and David on Oct 21
+
+o Integrate new service fingerprint submissions (we have more than
+ 2,531 submissions in two files since 11/30/10)
+
+o Integrate new OS detection submissions (1,893 since 6/22/11)
+
+o Add options in configure script for users to specify where to find
+ subversion lib/include dirs (like we do with our other library
+ dependencies). See this mail:
+ http://seclists.org/nmap-dev/2012/q1/37
+ -- David added --with-apr and --with-subversion
+
+o We need to fix the svn server so that Nmap committers can make
+ branches from /nmap to /nmap-exp. We may need to add some sort of
+ OPTIONS permission to the root directory or something, because
+ they're getting errors like:
+ $ svn cp https://svn.nmap.org/nmap https://svn.nmap.org/nmap-exp/branchname
+ svn: Server sent unexpected return value (403 Forbidden) in response
+ to OPTIONS request for 'https://svn.nmap.org'
+ - Patrick also reported some other funny business related to svn
+ mv'ing directories in email to Fyodor and David.
+
+o Give CPE visibility to NSE.
+ - done by Henri
+
+o Document the new IPv6 OS detection novelty system in os-detection.xml
+
+o Do more thinking/researching/investigating the way our machine
+ learning IPv6 OS detection system decides whether a match is perfect
+ and/or how close the match is. Maybe our current system works well
+ enough, we'll need to watch how it performs as we increase the DB
+ size and collect/integrate more signatures. The goal is to:
+ o Producing fewer way-off matches since it would have a way (like our
+ current system) to decide how close the match really is
+ o Doing a better job about printing fingerprints for matches with
+ aren't close enough
+
+o Improve the "run Zenmap as root" menu item to work on distributions
+ without su-to-root. We might even want to improve Zenmap so that it
+ itself does not have to run as root, and just executes Nmap that
+ way. Rather than not showing Zenmap as root on the Menu of
+ non-working systems, it might be better to have it but let it give
+ an error message (and then, perhaps, run as nonroot) so that users
+ of those distributions are more likely to contribute a fix. We also
+ might want to look at how the distributions themselves package Zenmap.
+
+o Consider changing Nsock so that it is able to take advantage of more
+ modern interfaces to dealing with large sockets, rather than just
+ select. Perhaps we should look at poll(), Windows completion ports,
+ and some of the advanced Linux APIs. Select() limits us to
+ descriptors no higher than FD_SETSIZE, and it may not performa all
+ that well. We should do some benchmarking and decide on the
+ interface to use for each platform. May want to take a look at
+ libevent (http://www.monkey.org/~provos/libevent/) for inspiration.
+ The libevent home page has some interesting benchmark graphs too.
+ [Josh implemented poll as a SoC student, but it had problems with
+ Nsock's architecture. O(1) lookups were becoming O(n) because of
+ the nature of the data structures. It was slower in his benchmarks.
+ Nsock would have change from a model of "loop over the event list,
+ and check to see if the fd for each event is set," to one of "loop
+ over the fd list, and see if there is a corresponding event for
+ each. It is the "see if the fd is set" operation that's O(1) with
+ select (it's FD_ISSET) and O(n) with poll (it's a traversal of a
+ linked list).]
+ o Henri added nsock-engines
+
+o Consider an update feed system for Nmap which let's people obtain
+ the latest Nmap data files, such as NSE scripts/libs, nmap-os-db,
+ nmap-service-probes, etc.
+ o Note that some scripts require updated compiled libraries. We
+ will need some sort of compatability system.
+ o One approach is "svn up". Note that Metasploit uses that approach
+ even for Windows by shipping .svn directories and an svn executable
+ with the Windows installer. In taht case we might need to have a
+ separate branch for each release that gets updated version/OS
+ databases and scripts.
+ o Another approach is a special feed system as is used by Nessus and
+ OpenVAS. OpenVAS uses a script wrapper around rsync, or an HTTP
+ download if that fails.
+ o Colin's analysis of different methods:
+ http://seclists.org/nmap-dev/2011/q2/821
+
+o [NSE] Consider using .idl files rather than manually coding all the
+ MSRPC stuff. The current idea, if we do this, is to have an
+ application in nmap-private-dev which converts .idl files to LUA
+ code for nmap/nselib. Consider adapting the pidl utility from Samba.
+ o Drazen did some work on this during SoC.
+ https://svn.nmap.org/nmap-exp/drazen/nmap-msrpc could get someone
+ started.
+ o We moved this out of the active section of the TODO because, while
+ it is still a good idea and we'd welcome the change if someone wants
+ to take it on, it isn't something that we are likely to make
+ progress on unless someone steps forward.
+
+o Implement a solution for people who want NIST CPE OS detection
+ results (we'll save version detection for a 2nd phase). Notes:
+ David report on CPE for OS Detection:
+ http://seclists.org/nmap-dev/2010/q3/278
+ David report on CPE for version detection:
+ http://seclists.org/nmap-dev/2010/q3/303
+ Nessus has described their integration of CPE:
+ http://blog.tenablesecurity.com/2010/05/common-platform-enumeration-cpe-with-nessus.html.
+ Older messages about it:
+ http://seclists.org/nmap-dev/2008/q4/627
+ http://seclists.org/nmap-dev/2010/q2/788
+
+o [NSE] HTTP spidering library/script
+
+o We should probably modify stdnse.get_script_args so that it first
+ checks [scriptname].[argname] and then (if that fails) looks for
+ [argname] by itself. This way people who are only running one
+ script or who want to use the same value for multiple scripts that
+ take the same argument can just give [argname]. But those who want
+ an argument to only apply to a specific script can give
+ [scriptname].[argname].
+ o The code is in place now, we just need to document the feature.
+
+o Script review
+ o Martin Swende patch to force script run
+ http://seclists.org/nmap-dev/2010/q4/567
+ o applied
+ o irc-info patch. http://seclists.org/nmap-dev/2011/q2/289.
+ o applied
+ o http-slowloris. http://seclists.org/nmap-dev/2011/q1/916.
+ o Had some issues--never got to a state ready for integration
+ o http-phpself-xss
+ - Would need to be rewritten to use newer spider.lua. Added an item
+ to incoming section of Nmap Script Ideas secwiki page.
+
+o Make new SecTools.Org site with the 2010 survey results.
+
+o Collect many more IPv6 OS detection training samples from users
+ - Can start with nmap-dev, but will probably have to do an Nmap
+ release too.
+
+o Integrate more NSE scripts, I think our review queue is getting
+ pretty long.
+
+o Decide what to do with Henri's nsock-engines branch
+ (/nmap-exp/henri/nsock-engines).
+
+o finish making nmap-update part of the nmap windows compile-time
+ infrastructure
+ o See if we can build just one project within a solution, rather
+ than having special "with nmap-update" configuration.
+
+o Add homedir support to Nmap for the updater
+
+o Fix expiration date parsing on Nmap Windows for the updater
+
+o Updater: Make a missing nmap-update.conf nonfatal (perhaps doesn't
+ even need to mention it).
+
+o Updater: Clean up the output messages (e.g. only print what user needs to see
+ unless debugging is specified)
+
+o [Nping] The --safe-payloads option should be default (though we
+ should keep it for backward compatability). We could then introduce
+ --include-payloads for cases where they are desired.
+
+o A program to canonicalize and tidy nmap-service-probes.
+ o Order of fields: m p v i d o h cpe:/a cpe:/h cpe:/o.
+ o Check for duplicate templates (except cpe:).
+ o Check for unknown templates.
+ o Canonicalize delimiters (use // first, otherwise try in order
+ | % = @ #).
+ o Retain line breaks and comments.
+
+o Document IPv6 OS detection at https://nmap.org/book/osdetect.html
+
+o Script review:
+ - New scripts from Paulino: http-wordpress-brute and http-joomla-brute,
+ http-majordomo2-dir-traversal.nse, http-trace, http-waf-detect
+ - http-methods patch. http://seclists.org/nmap-dev/2011/q1/936.
+ - quake3-info. http://seclists.org/nmap-dev/2011/q2/172.
+ - smb-os-discovery additional
+ information. http://seclists.org/nmap-dev/2011/q2/276.
+ - Outlook web
+ address. http://seclists.org/nmap-dev/2011/q2/296. [probably not
+ going to merge to Nmap trunk at this point, though it is good that
+ the script is available for d/l for those who need it. ]
+
+o Fix reported (by many people) crash when trying to launch Zenmap on
+ Mac OS X 10.7 (Lion).
+
+o Unless we get good arguments for keeping it, we should remove Mac OS
+ X PowerPC support from our binaries. Apple stopped selling PowerPC
+ machines in 2006 and they stopped making new OS releases available
+ for PowerPC as of Snow Leopard (10.6) in August 2009. See this
+ thread: http://seclists.org/nmap-dev/2011/q3/430
+
+o Improvements to the Nmap multicast IPv6 host discovery scripts
+ - Note that we hope to move them into core Nmap at some point, but
+ would be good to improve them for now.
+ - They should probably print the discovered IPv6 addresses, otherwise
+ they don't actually give the user any information (despite doing
+ their work) unless you give the newtargets script arg. This would
+ be similar to the current behavior of broadcast-ping.
+ - It might be nice if they gave the target MAC address and vendor
+ when printing the discovered IPv6 information too. Daniel Miller
+ wrote an initial patch for this (though we need to make sure it can
+ handle (e.g. doesn't crash for) non-ethernet
+ devices:http://seclists.org/nmap-dev/2011/q3/862. Our broadcast-ping script
+ currently prints MAC addresses.
+ - It is great that the scripts properly use a specific device when
+ given the Nmap -e option, but they shouldn't require this. They
+ should do something smart if no specific device name is given.
+ Examples include performing on all compatable devices or trying to
+ pick the best device. The all-devices appraoch may be the best,
+ IMHO. That is how our broadcast-ping script works now.
+
+o Add anti-spam defenses to secwiki.com to stop the current onslaught
+ of spam. An extention like ConfirmEdit
+ (http://www.mediawiki.org/wiki/Extension:ConfirmEdit) may be a good choice.
+
+o Collect a bunch of IPv6 OS detection signatures from users,
+ integrate them, and then when we have enough, re-enable OS detection
+ results.
+
+o IPv6 OS detection working (when run on) Solaris and AIX
+ - AIX 6.1 - iSeries / System p
+ - AIX 7.1 - iSeries / System p
+ - Solaris 10 - SPARC
+
+o We should consider splitting a 'brute' category out of the 'auth'
+ category now that we have so many brute force scripts. I suppose
+ users can already do "--script *-brute", but having its own category
+ might still be nice.
+
+o IPv6 OS detection merge
+ o [DONE] Initial branch working (nmap-exp/luis/nmap-os6)
+ o [DONE] Implement the 2 remaining probes
+ o [DONE] Disable the printing of matches (except maybe with debug on). We
+ want more training examples first so that results are better.
+ o [DONE] Merge to /nmap
+
+o Document Nmap CPE support in appropriate places (candidates:
+ refguide, os detection book chapter, version detection book chapter,
+ output book chapter).
+
+o Finish CPE support code
+ - Escape certain values that can be inserted into cpe string through
+ substitution, like cpe:/a:apache:httpd:$1 where $1 contains a
+ colon.
+
+o Add advanced IPv6 host discovery features
+ o Initially done using NSE by adding these scripts:
+ targets-ipv6-multicast-slaac, targets-ipv6-multicast-invalid-dst, and
+ targets-ipv6-multicast-echo
+
+o Initial IPv6 OS detection system (may not make it into stable
+ though, but we want to at least have it working in a branch first.)
+ - OK, it is working in nmap-exp/luis/nmap-os6
+
+o Investigate a probe/response matching problem reported by QA Cafe
+ Matthew Stickney and Joe McEachern of QA Cafe. See this thread:
+ http://seclists.org/nmap-dev/2011/q3/227
+
+o When our winpcap installer is run in silent mode
+ (e.g. "winpcap-nmap-4.12.exe /S"), it seems to execute nmap.exe if
+ that binary exists in the same directory. This leads to a cmd.exe
+ window briefly poping up as Nmap displays its console help output.
+ Moving the Winpcap installer into its own subdir and running it from
+ there seems to fix this (because it then can't find nmap.exe to
+ run), but it would be better to determine why this is happening in
+ the first place and fix it.
+
+o Obtain Nmap data directory information from nmaprc at runtime rather than
+ compiled in -- among other advantages this is needed to make
+ relocateable rpm. [actually we ended up doing this without needing
+ nmaprc for now]
+
+o Summer of Code feature creeper:
+ o Ncat should probably have an --append-output option like Nmap does
+ so that we can use -o without clobbering existing file. This would
+ at least be useful for chat.nmap.org.
+ o Change Zenmap bug reporter so that instead of an automatic
+ submission system, we print a stack trace and request that the user
+ send a bug report to nmap-dev.
+
+o [Ncat] Solve a crash that only happens on Windows when connecting
+ with --ssl-verify and -vvv, for example
+ ncat --ssl-verify -vvv www.amazon.com 443
+ The crash happens in the function verify_callback, when the function
+ X509_NAME_print_ex_fp is called. Just commenting those two calls
+ avoids the problem. By trying different combinations of debug print
+ statements, I once got the message
+ OPENSSL_Uplink(10109000,08): no OPENSSL_Applink
+ This refers to a Windows dynamic linking issue:
+ http://www.openssl.org/support/faq.html#PROG2
+ However I tried both including <openssl/applink.c> and changing the
+ linker mode to /MD, and neither changed the behavior.
+ Changing the flags from XN_FLAG_ONELINE to 0 seems to make the
+ problem go away.
+
+o Integrate new OS detection submissions (We have about 1,700
+ submissions since 11/30/10)
+
+o Nmap should defer address parsing in arguments until it has read
+ through all the args. Otherwise you get an error if you use like -S
+ with an IPv6 address before you put -6 in the command line. You get
+ a similar problem if you do "-A -6" (but "-6 -A works properly).
+ This is a possible feature creeper task.
+
+o Ncat chat (at least in ssl mode) no longer gives the banner greeting
+ when I connect. This worked in r23918, but not in r24185, which is
+ the one running on chat.nmap.org as of 6/20/11. Verify by running
+ "ncat --ssl -v chat.nmap.org"
+
+o IPv6 Neighbor Discovery-based host discovery (analog to ARP scan).
+
+o Investigate and document how easy it is to drop Ncat.exe by itself
+ on other systems and have it work. We should also look into the
+ dependencies of Nmap and Zenmap. It may be instructive to look at
+ "Portable Firefox"
+ (http://portableapps.com/apps/internet/firefox_portable) which is
+ built using open source technology from portableapps.com, or look at
+ "The Network Toolkit" by Cace
+ (http://www.cacetech.com/products/network_toolkit.html). For Nmap
+ and Nping, we may want to improve our Winpcap to load as a DLL
+ without requiring installation. There is a separate TODO item for that.
+
+o The SCRIPT_NAME variable should not include the ".nse" in script
+ names. Currently, it omits that for scripts in the DB, but includes
+ it for scripts you specify based on their filename. See:
+ http://seclists.org/nmap-dev/2011/q2/481
+
+o If possible, Ncat, in listen mode, should probably listen on the system's
+ IPv6 interfaces as well as IPv4. This is what servers like apache
+ and ssh do by default. It might now be possible to listen on IPv6
+ by running a second ncat with -6, but that doesn't really work for
+ broker and chat modes because you want the IPv6 users to be able to
+ talk to IPv4 and vice versa.
+ - This was partially implemented, but still doesn't seem to work in
+ --chat mode. Can test against chat.nmap.org
+ - Done. Tested on scanme with David & Fyodor on 7/18/11.
+
+o Right before the release, we could build Ncat portable and post it
+ on https://nmap.org/ncat/.
+ - Actually we did that for 5.59BETA1, which is good enough for now.
+
+o CHANGELOG updates [Fyodor]
+
+o [Ncat] Add new certificate bundle (ca-bundle.crt) since the current
+ one is out of date. See http://seclists.org/nmap-dev/2011/q2/641.
+
+o Move these prerule/postrule script ideas to secwiki script idea page
+ if appropriate (with a bit more details):
+ o AS Number to IP ranges: http://seclists.org/nmap-dev/2010/q2/101
+ In progress.
+ o DNS service discovery (Bonjour): http://en.wikipedia.org/wiki/Bonjour_%28software%29
+ Present as dns-service-discovery.nse.
+ o Netbios Name Service
+ Already present as broadcast-netbios-master-browser.nse?
+ o DHCP broadcast requests
+ Present as dhcp-discover.nse.
+ o Postrules could be created which give final reports/statistics or
+ other useful output. Like a reverse-index, which shows all the open
+ port numbers individually and the hosts which had that port open
+ (e.g. so you can see all the ssh servers at once, etc.)
+ Admittedly you can do that pretty easy with Zenmap instead.
+ Have a few of these: ssh-hostkey and upcoming creds-summary.
+ o We could have a prerule sniffer script which uses pcap to sniff
+ traffic for some short configurable amount of time and then adds the
+ discovered hosts to the target list.
+ Already present as targets-sniffer.nse.
+ o We could have a script which takes traceroute results and adds them to the target list.
+ Already present as targets-traceroute.nse.
+
+o [NSE] Add these ideas to secwiki script ideas page if appropriate
+ (with a bit more details):
+ o Windows system logs (like sysinternals' psloglist)
+ o Services (like sysinternals' psservice)
+ o A script (or modification to smb-check-vulns) to
+ detect this MSRPC vulnerability:
+ http://seclists.org/fulldisclosure/2010/Aug/122
+ o BasicHTML/XML parser library? For example, Sven Klemm wrote a script
+ which uses libxml2: http://seclists.org/nmap-dev/2008/q3/0462.html.
+ And here is one by Duart Silva using Expat:
+ http://seclists.org/nmap-dev/2009/q3/1093.
+ o Add detection of duplicate machines via IP.ID technique.
+ Maybe I should use uptime timestamps too. Oh, and MAC addresses
+ too. Our SSH host key script is useful for this as well.
+
+o Look into iplog ( http://ojnk.sourceforge.net/ ) -z option which is
+ supposed to fool OS detection.
+ o The software is no longer maintained, so we're not going to worry
+ about it. The page says: "I am through working on this project. I
+ will not be making any updates, and I will ignore just about all
+ email about it. If anybody wants to take it over (for whatever
+ reason), let me know"
+
+o [NSE] Consider how we compare to the Nessus Web Application Attack
+ scripts
+ (http://blog.tenablesecurity.com/2009/06/enhanced-web-application-attacks-added-to-nessus.html).
+ [Joao making a list of web scripts which we might find useful,
+ Fyodor asking HD moore for permission to use http enum dir list]
+
+o [NSE] HTTP persistant connections/keepalive? May make
+ spidering/grinding/auth cracking more efficient
+
+o [NSE] HTTP Pipelining support? May make spidering/grinding/auth
+ cracking more efficient
+
+o [NSE] HTTP Cookie suppport? Might be useful for spidering sites which use it
+ for authentication/authorization/personalization.
+
+o [NSE] URL grinder checks for existence of applications in common/default
+ paths. Scanning http paths to see if they exist is in some ways
+ similar to scanning to see which ports are open.
+ o Our http-enum does this.
+
+o Investigate why and whether we need mswin32/pcap-include/pcap-int.h.
+ This file is not included in the official WinPcap 4.1.1 developers'
+ pack
+ (http://www.winpcap.org/install/bin/WpdPack_4_1_1.zip). Presumably
+ it covers internal functions and structures which we aren't really
+ supposed to access it. If we can get rid of it, that would be
+ great. If we need it, we should probably upgrade to the
+ 4.1.1. version (presumably from the Winpcap source code
+ distribution). Right now it is included in tcpip.h,
+ nsock/src/nsock_pcap.h, and nping/common_modified.cc: o David looked
+ into it. He says it isn't distributed with the WinPcap developer's
+ pack. You have to extract it from the source file. He updated to the
+ 4.1.1 version. He says The entire reason we need it is so we can
+ peek at the definition of struct pcap, so we can access the
+ pcap.adapter member on Windows. In order to pass it to
+ PacketSetReadTimeout. Usually struct pcap is an opaque type and you
+ are only supposed to access it through a pcap_t *. Unfortunately I
+ don't think there's an easy way to manipulate the timeouts in
+ WInPcap like we do on other platforms. You can specify a timeout
+ when you do pcap_open, but we like to set a timeout on every
+ read. So we sort of sneak in and call PacketSetReadTimeout. In the
+ code there's even a comment: "BUGBUG: This is cheating." libdnet
+ also uses the Packet* functions, but in a more innocuous
+ way. It doesn't access them through a struct pcap, so it
+ doesn't need pcap-int.h. David tried testing whether this makes
+ any signficiant difference--to see if we could just remove the
+ PcapSetReadTimeout()--but that didn't work out.
+ - We're not going to worry about this for now since it isn't
+ important enough to pester the pcap people about, and they don't
+ seem to be changing their internal structure anyway. And if they
+ do, we can get the new pcap-int.h.
+
+o Further brainstorm and consider implementing more prerule/postrule
+ scripts:
+ o [Implemented] dns-zone-transfer
+ o [Implemented, but a joke] http-california-plates
+
+o Investigate this interface-matching problem on Windows:
+ http://seclists.org/nmap-dev/2011/q1/52. It is related to the
+ libdnet changes we made to allow choosing the correct physical
+ interface when teamed interfaces share the same MAC.
+ I think this is solved with the rewritten libdnet code (that uses
+ GetAdaptersAddresses) in my nmap-ipv6 branch. --David
+
+o [Ncat] When in connection brokering or chat mode with ssl support
+ enabled, if one client connects and doesn't complete ssl negotiation,
+ it hangs any other connections while that first is active. One way to
+ reproduce:
+ Run SSL chat server like: /usr/local/bin/ncat --ssl -l --chat
+ Window #1: Connect without ssl: ncat -v chatserverip
+ Window #2: Try to connect with SSL: ncat -v --ssl chatserverip
+ Window #2 will not work while #1 is active. If you quit #1, #2
+ should work again.
+
+o IPv6 todo.
+ - Protocol scan (-sO).
+
+o [Ncat] Find out what RDP port forwarding apparently doesn't work on
+ Windows. http://seclists.org/nmap-dev/2011/q1/86
+
+o Add raw packet IPv6 support, initially for SYN scan
+ o After that can add UDP scan, and sometime OS detection (David did
+ some research on what IPv6 OS detection might require).
+
+o When I (Fyodor) scan scanme.nmap.org with the command "nmap -sC -p80
+-Pn -n scanme.nmap.org", I get a blank http-favicon line like:
+ 80/tcp open http
+ |_http-title: Go ahead and ScanMe!
+ |_http-favicon:
+ But if I use "--script http-favicon" instead of -sC, it works fine.
+
+o UDP scanning with IP options causes "Received short ICMP packet" on
+ receipt. http://seclists.org/nmap-dev/2011/q1/82
+
+
+o [Zenmap] Make formerly open ports that are now closed or filtered
+ disappear from the "Ports / Hosts" tab. This appears to be related
+ to ignored states; if in the second scan I use -d2 so all ports are
+ included in the output, the interface is updated correctly.
+ http://seclists.org/nmap-dev/2010/q4/659
+
+o [Zenmap] When a target is unresponsive (and its distance isn't
+ known), put it at the next furthest ring from the known traceroute
+ hosts (with a dashed line), instead of putting it at the first ring.
+ See http://seclists.org/nmap-dev/2011/q1/834.
+
+o Rewrite the portreasons code not to use parallel arrays
+ (reason_text, reason_pl_text) and not to require special alignment
+ between the enum codes and (for example) ICMP types. Instead define
+ one structure containing all relevant information about a reason,
+ and define helper functions to map ICMP types to reason codes. In
+ particular, code like this needs to go away: current_reason =
+ ping->type + ER_ICMPTYPE_MOD; if (current_reason == ER_DESTUNREACH)
+ current_reason = ping->code + ER_ICMPCODE_MOD;
+
+o Fix memory consumption problem in drda-info (see
+ http://seclists.org/nmap-dev/2011/q2/451)
+ - Fixed (turned out to affect a lot of scripts)
+
+o Script dispensation
+ - sip-enum-users and
+ sip-brute. http://seclists.org/nmap-dev/2011/q2/56.
+ o Merged
+ - xmpp. http://seclists.org/nmap-dev/2011/q2/239.
+ o Merged
+
+o Script review/disposition:
+ - Merged: DNSSEC enumeration. http://seclists.org/nmap-dev/2011/q1/406.
+ - Merged: quake3-master-getservers patch. http://seclists.org/nmap-dev/2011/q1/925.
+ - Merged: backorifice-info. http://seclists.org/nmap-dev/2011/q2/185.
+ - Merged: omp2-brute and omp2-enum-targets. http://seclists.org/nmap-dev/2011/q2/231.
+ - Merged: http-wp-plugins. http://seclists.org/nmap-dev/2011/q1/806.
+
+o Decide what to do about ms-sql-info slowing scans:
+ http://seclists.org/nmap-dev/2011/q1/913
+ - patch applied: http://seclists.org/nmap-dev/2011/q1/1102
+
+o Script disposition
+ - Patch to get interfaces by Djalal.
+ http://seclists.org/nmap-dev/2011/q1/291
+ - Incorporated
+ - epmd-info. http://seclists.org/nmap-dev/2011/q1/931.
+ - Incorporated
+ - google-id. http://seclists.org/nmap-dev/2011/q1/952.
+ - Incorporated as http-affiliate-id
+
+o [Ndiff] should, in non-verbose mode, perhaps not print the changed
+ Nmap version and/or scan time if nothing else has changed between
+ two files. See http://seclists.org/nmap-dev/2011/q1/674.
+
+o Script review disposition:
+ - ssl-known_key http://seclists.org/nmap-dev/2010/q4/733
+ Thread continues at http://seclists.org/nmap-dev/2011/q1/26.
+ - Merged
+ - dns-nsec-enum
+ - Merged
+
+o The file /nmap/mswin32/icon1.ico is used by the NSIS installer to
+ set the Nmap uninstall icon (I'm not sure if it is used for anything
+ else). But this is a very old icon and doesn't match the blue eye
+ we use now. So we should probably update that with a modern "blue
+ insecure eye" icon. I (Fyodor) tried simply replacing icon1.ico
+ with http://insecure.org/shared/images/tiny-eyeicon.ico, but that
+ didn't work. It must not meet the required format.
+
+o Add some content to https://secwiki.org and announce it.
+
+o Removing -sR option (but keeping the functionality as part
+ of -sV). See http://seclists.org/nmap-dev/2011/q1/688
+ - Update Nmap documentation/book to remove it there too
+
+
+o Script disposition:
+ - dns-brute by cirrus. http://seclists.org/nmap-dev/2011/q1/351
+ Should share domain list with http-vhosts.
+ git://code.0x0lab.org/nmap-dns-brute.git
+ - Added by David
+
+o Write and post 2010 SoC Successes writeup [Fyodor]
+
+o Script review
+ - quake3-master-getservers http://seclists.org/nmap-dev/2011/q1/64
+ [merged]
+ - dpap-brute by Patrik Karlsson.
+ http://seclists.org/nmap-dev/2011/q1/252.
+ [merged]
+
+o The -V option to Nmap, in addition to reporting the version number,
+ should give details on how Nmap was compiled and the environment it
+ is running on. This includes things like whether SSL is enabled,
+ the platform string, versions of libraries it is linked to, and
+ other stuff which is often useful in debugging problems.
+ o We want to list at least:
+ o Nmap version number (that line is fine as is)
+ o host platform string (for which it was compiled)
+ o Whether OpenSSL and LibSSL, NLS, and IPv6 are enabled
+ - Version number of OpenSSL and LibSSL if those are enabled
+ o Version numbers of libdnet, libpcre, and libpcap
+
+o Script review:
+ - SCADA scripts http://seclists.org/nmap-dev/2010/q4/612
+ http://seclists.org/nmap-dev/2010/q4/613
+ http://seclists.org/nmap-dev/2010/q4/623
+ http://seclists.org/nmap-dev/2010/q4/639
+ [on hold]
+ - servicetags http://seclists.org/nmap-dev/2010/q4/691
+ needs new testing on OpenSolaris: http://seclists.org/nmap-dev/2011/q1/91
+ [committed]
+ - firewalk-path http://seclists.org/nmap-dev/2011/q1/63
+ [committed over previous firewalk script]
+ - snmp-ios-config http://seclists.org/nmap-dev/2011/q1/10
+ Requires a TFTP server; decision was to build such server in Lua
+ if possible. Patrik Karlsson's beginning TFTP implementation:
+ http://seclists.org/nmap-dev/2011/q1/169.
+ [committed by Patrik]
+
+o Script merged: p2p-dropbox-listener
+ http://seclists.org/nmap-dev/2010/q4/689
+
+o A trivial change: we currently print some lines about NSE
+ pre-scanning and post-scanning in verbose mode even when no such
+ scripts are being run. We should not print those in that case. For
+ example, nmap -A -v scanme.nmap.org gives me these superfluous lines:
+ NSE: Script Pre-scanning.
+ NSE: Starting runlevel 1 (of 2) scan.
+ Initiating NSE at 12:23
+ Completed NSE at 12:23, 0.00s elapsed
+ NSE: Starting runlevel 2 (of 2) scan.
+ NSE: Script scanning 64.13.134.52.
+ NSE: Starting runlevel 1 (of 2) scan.
+ Initiating NSE at 12:24
+ Completed NSE at 12:24, 4.14s elapsed
+ NSE: Starting runlevel 2 (of 2) scan.
+ NSE: Script Post-scanning.
+ NSE: Starting runlevel 1 (of 2) scan.
+ NSE: Starting runlevel 2 (of 2) scan.
+
+o Do new Nmap release with the stuff merged from SoC students and
+ other new developments.
+
+o Modify Zenmap to use the new --script-help system to enumerate
+ scripts and collect information such as their descriptions. This
+ will resolve the problem of Nmap's broadcast prerule scripts running
+ when you open the profile editor.
+
+o Document --script-help in docs/refguide.xml and docs/scripting.xml.
+
+o [Zenmap] Brian Krebs found a problem (which Fyodor is able to
+ reproduce) in the target selector on the left pane. When you select
+ one of the scanned targets, it is supposed to jump to that target in
+ the "Nmap Output" tab on the right pane. Instead, nothing seems to
+ happen. One of our output format changes probably broke the
+ feature. It still works fine if you have the "Ports / Hosts" or
+ "Host Details" tabs active in the right pane instead.
+
+o Include a --script-help system to Nmap, which provides user readable
+ text help and also machine parsable XML information for scripts
+ which match a pattern (e.g. the same sort of arguments you could use
+ for --script, like a category or http-* or whatever). The
+ --script-help ONLY provides help and quits, it does not run the
+ script. For some initial implementation work, see this thread:
+ http://seclists.org/nmap-dev/2011/q1/163
+
+o [Nping] See whether --echo-client mode really requires root, and
+ remove that restriction if not.
+ Luis explanation for requiring root:
+ http://seclists.org/nmap-dev/2011/q1/248
+
+o Script review:
+ - p2p-dropbox-listener http://seclists.org/nmap-dev/2010/q4/689
+
+o Decide whether to include NSE console script help, decide on
+ implementation issues. http://seclists.org/nmap-dev/2011/q1/163
+
+o [Zenmap] Use a more efficient algorithm to update the display of Nmap normal
+ output in live scans.
+ zenmapGUI.NmapOutputViewer.NmapOutputViewer.refresh_output calls
+ zenmapCore.NmapCommand.NmapCommand.get_output, which re-reads the
+ entire output file (into memory) and then puts it in the text buffer
+ if it has changed. So already we're storing the whole output twice in
+ memory. When the text field changes, update_output_colors
+ re-highlights the whole file.
+
+o Update changelog to note recent changes
+
+o Do final dev/test release
+
+o If Nping is compiled w/o SSL support, and the user specifies an
+ encryption key, it should fail and insist they use --no-crypto
+ rather than ignoring the key and omitting crypto. Otherwise the
+ user might think they're getting encryption when they're not. David
+ found this problem in the server, and we also should check how the
+ client behaves.
+
+o [Ncat] Make --exec work in conjunction with --proxy. The --proxy
+ code path skips the --exec code. See
+ http://seclists.org/nmap-dev/2010/q4/604 and the test "--exec
+ through proxy" in ncat-test.pl.
+
+o Decide what to do about Nmap static binaries failing to work on new
+ Fedora releases (and others?). See these threads:
+ http://seclists.org/nmap-dev/2011/q1/46 and
+ http://seclists.org/nmap-dev/2010/q1/308
+ o We ended up dynamically linking system libs in the RPM rather than
+ statically linking them. We still statically link things like lua,
+ pcre, ssl, etc.
+
+o Fix our mac builds so that they contain SSL support again (5.35DC1
+ did, but TEST1 and TEST2 didn't for some reason.
+
+o Add our broadcast discovery scripts to a "broadcast" category (they
+ should generally just be in "broadcast" and (assuming they are safe)
+ "safe", and not normal "discovery". Update scripting.xml to note
+ this new category too.
+
+o The latest IANA services file
+ (http://www.iana.org/assignments/port-numbers) has many identified
+ services which are still "unknown" in our files because ours is
+ based on a much older version of that file. We should probably take
+ that file and add names and comments to our nmap-services-all where
+ they are "unknown" in our file. An example of such a port is 3872,
+ oem-agent.
+
+o Script review:
+ - patch for ftp-proftpd-backdoor
+ http://seclists.org/nmap-dev/2010/q4/678
+ - patch for hddtemp-info http://seclists.org/nmap-dev/2010/q4/676
+
+o We should probably update our Windows build systems to use Python
+ 2.7. As of 11/8, it looks like all our dependency libraries are
+ available for 2.7:
+ o David upgraded and it worked, though Rob found a potential problem
+ and added vcredist 2008. Fyodor will test on the official Win7 Nmap
+ build system.
+ PyGTK: 2.22.0 IS available for 2.7
+ PyCairo: 1.8.10 IS available for 2.7
+ PyGObject: 2.26.0 IS available for 2.7
+ Py2exe: 0.6.9 IS available for 2.7
+
+o Do service/version detection submission integration (last done in
+ April)
+
+o Do os detection submission integration (last done in April)
+
+o Script review:
+ - modbus-enum http://seclists.org/nmap-dev/2010/q4/489
+
+o Create Nmap wiki
+ o Decide on domain name
+ o Include insecure Chrome
+ o Decide on wiki software, probably just use mediawiki
+ o install it on a Linode, probably Web
+
+o [NSE] Web application fingerprinting script. Would be great to be
+ able to take a URL and determine things like "this is Joomla" or
+ "this is Plone" or "Mediawiki" or whatever. Rather than hard code
+ regular expressions or other tests in a script, it should use a
+ signature file like Nmap OS and version detection do. Might work in
+ combination with URL grinder to check for applications at
+ default/common locations. See also a script that does favicon
+ scanning TODO item.
+ - http-enum pretty much does this now.
+
+o Update our distribution build systems and documentation to use
+ Visual C++ 2010 Express rather than the 2008 version. See
+ http://www.microsoft.com/express/Windows/
+
+o Dependency licensing issues (OpenSSL, Python, GTK+, etc.)
+ o Almost done! We just have some file renaming/organizing left to do.
+ o We should do an audit to ensure that we are in complete compliance for the
+ licenses of all the software we ship in any of our downloads, as some
+ licenses have special clauses for things like including their
+ license/copyright file, mentioning them in our documentation, etc.
+ And of course we want to credit them properly even where the license
+ doesn't require it. We should probably make a list of these in our
+ docs/ directory along with any special information/requirements of
+ their license. And maybe we should put the current licenses in a
+ subdir too. In particular, these come to mind:
+ o libpcre
+ o lua
+ o OpenSSL
+ o libpcap
+ o GTK+/Glib/ATK/Pango/PyGTK (Win/Mac versions of Zenmap link to
+ PyGTK)
+ o SQLite
+ o Python (Win/Mac versions of Zenmap link to Python)
+ o X.org libraries (Mac version links to them)
+ o libdnet
+
+o Small NSEDoc bug:
+ https://nmap.org/nsedoc/scripts/dns-zone-transfer.html contains 'id
+ \222\173' near the bottom. This is presumably due to misparsing this
+ line from the script: local req_id = '\222\173'. Given that we don't
+ use IDs any more, maybe we can just get rid of the functionality.
+
+o [NSE] We should probably enable broadcast scripts to work better by
+ (initial thoughts):
+ o Done and merged by David!
+ 1) Change NSE to always set nsp_setbroadcast() on new sockets
+ 2) Change nsock to create real sockets at time of nsi_new so you can
+ bind to them.
+ See this thread (only some of the messages involve broadcast
+ support): http://seclists.org/nmap-dev/2010/q3/357
+
+o [NSE] Review scripts:
+ o Hostmap (Ange Gutek) - http://seclists.org/nmap-dev/2010/q3/159
+
+o Post BH/Defcon Nmap videos
+
+o Let Nsock log to stderr, so its messages don't get mixed up with the
+ output stream when Ncat is run with -vvv.
+ http://seclists.org/nmap-dev/2010/q3/113
+
+o [NSE] Our http-brute should probably support form POST method rather
+ than just GET because some forms require that.
+
+o Nping needs to call nsp_delete so that its socket descriptors are
+ not left behind.
+
+o [Zenmap] Add a button to select script files from the filesystem.
+
+o [Zenmap] Show help for individual script arguments in the Help pane,
+ not for all arguments at once.
+
+o Upgrade our Windows OpenSSL binaries from version 0.9.8j to the
+ newest version (1.0.0a as of Aug 12, 2010).
+
+o Since Libdnet files (such as ltmain.sh) are apparently only used by
+ libdnet (they used to be used by shared library NSE C scripts), we
+ should move them to the libdnet directory.
+ o Turned out to be a pain. See
+ http://seclists.org/nmap-dev/2010/q3/733
+
+o [Zenmap] Consider a memory usage audit. This thread includes a claim
+ that a 4,094 host scan can take up 800MB+ of memory in Zenmap:
+ http://seclists.org/nmap-dev/2010/q1/1127
+ The reporter mentioned Guppy/Heapy to debug memory use:
+ http://guppy-pe.sourceforge.net/
+ http://www.pkgcore.org/trac/pkgcore/doc/dev-notes/heapy.rst. Many
+ Nmap survey respondants complained about this too.
+ Note: Fyodor has a 50MB scan log file named ms-vscan.xml which
+ demonstrates this problem. When trying to load the file, Zenmap
+ grows to 1150MB of RAM, pegs the CPU usage at 100% for many
+ minutes or maybe hours (I forgot about it, but woke up the next day
+ to find that it had started, was then using 2.4GB of RAM. The
+ hosts/services functionality seemed to work, although it would take
+ a minute or so to switch from say "ftp" port to view "ssh" ports.
+
+o [NSE] Maybe we should create a script which checks once a day
+ whether similar tools (Metasploit, Nessus, OpenVAS, etc.) have any
+ new modules, and then mails out a list of them with the description
+ fields. The mail could go to just interested parties, or maybe
+ nmap-dev. This may help prevent important vulnerabilities from
+ falling through the cracks. Perhaps we would include new NSEs in
+ there too, especially if we open it up as a public list.
+
+o Now that NSE has more script phases (prerule, postrule, hostrule,
+ portrule, and versionrule soon to come), the NSEDoc should specify
+ which phases a script belongs to.
+
+o Consider implementing a nsock_pcap_close() function or making
+ nsp_delete() call pcap_close() when pcap IODs are used. Currently valgrind
+ warns about a socket descriptor left opened (at least in Nping).
+ See http://seclists.org/nmap-dev/2010/q3/305.
+ o It turns out that the pcap descriptors are being closed properly,
+ but Nping isn't calling nsp_delete.
+
+o [NSE] High speed brute force HTTP authentication. Possibly POST and
+ GET/HEAD brute force cracking. [done except for form POST, adding
+ separate TODO item for that]
+
+o [NSE] Review scripts:
+ o New brute, vnc, and svn scripts by Patrik. This guy is a coding
+ machine :). http://seclists.org/nmap-dev/2010/q3/111
+ o rmi-dumpregistry by Martin
+ Swende. http://seclists.org/nmap-dev/2010/q2/904
+ o path-mtu.nse - http://seclists.org/nmap-dev/2010/q3/222
+ o 15 more from Patrik :). http://seclists.org/nmap-dev/2010/q3/284
+
+o [NSE] Consider modifying our brute force scripts to take advantage
+ of the new NSE multiple-thread parallelism features.
+ - We've done this with db2-brute, but the DB may have been a
+ bottleneck there, so we should probably do more testing after
+ modifying another script for this sort of parallel cracking.
+
+o Look into implementing security technologies such as DEP and ASLR on
+ Windows: http://seclists.org/nmap-dev/2010/q3/12.
+
+o Ncat and Nmap should probably support SSL Server Name Indication
+ (SNI). See this thread: http://seclists.org/nmap-dev/2010/q3/112.
+ We need this to talk to web servers which share one SSL IP and port
+ because we need to ask for the right SSL key.
+
+o [NSE] In the same way as our -brute scripts limit their runtime by
+ default, I think qscan should be less intense by default. For
+ example, perhaps it could run by default on no more than 8 open
+ ports, plus up to 1 closed port. Right now it does things like
+ running on 65,000+ closed ports and bloats scan time (and output).
+ Of course there could (probably should) still be options to enable
+ more intense qscanning.
+
+o [Web] We should see if we can easily put the Insecure chrome around
+ Apache directory listings and 404 pages (e.g. https://nmap.org/dist/
+ and https://nmap.org/404). I think we may have had this working
+ before the move to Linode, so maybe check conf/httpd.conf.syn.
+
+o Do a serious analysis if and how we should use the NIST CPE standard
+ (http://cpe.mitre.org/) for OS detection and (maybe in a different
+ phase) version detection results. One thing to note is that they
+ may not have entries for many vendors we have. For example, one
+ person told me they couldn't find SonicWall or D-Link in the CPE
+ dictionary. Here are some
+ discussions threads on adding CPE to Nmap:
+ http://seclists.org/nmap-dev/2008/q4/627 and
+ http://seclists.org/nmap-dev/2010/q2/788.
+ Nessus has described their integration of CPE at
+ http://blog.tenablesecurity.com/2010/05/common-platform-enumeration-cpe-with-nessus.html.
+
+o [NSE] Create NSE scripts to scan for and/or exploit these VXWorks issues:
+ http://blog.metasploit.com/2010/08/vxworks-vulnerabilities.html [Ron
+ may be able to do this. Or others are welcome to take a shot at it.]
+
+o The -g (set source port) option doesn't seem to be working (at least
+ in Fyodor's quick tests) for version detection or connect() scan,
+ and apparently doesn't work for NSE either. We should fix this
+ where we can, and document the limitation in the refguide where it
+ is impractical. Also see http://seclists.org/nmap-dev/2010/q2/576.
+
+o [Zenmap] script selection interface for deciding which NSE scripts to
+ run. Ideally it would have a great, intuitive UI, the smarts to
+ know the scripts/categories available, display NSEdoc info, and even
+ know what arguments each can take.
+
+o Review http-xst (Eduardo Garcia Melia) -
+ http://seclists.org/nmap-dev/2010/q3/159
+
+o [NSE] Investigate sslv2.nse falsely reporting SSLv2 as being
+ supported.
+ http://seclists.org/nmap-dev/2010/q2/754
+
+o [NSE] The NSEDoc for some scripts includes large "Functions"
+ sections which aren't really useful to script users. For example,
+ see https://nmap.org/nsedoc/scripts/snmp-interfaces.html. Perhaps we
+ should hide these behind an expander like "Developer documentation
+ (show)". I don't think we need to do this for libraries, since
+ developers are the primary audience for those documents.
+ o Talked to David. We should just remove the function entries.
+
+o We should add a shortport.http or similar function because numerous
+ services use this protocol and many of our scripts already try to
+ detect http in their portrule in inconsistent ways.
+
+o [NSE] Maybe we should create a class of scripts which only run one
+ time per scan, similar to auxiliary modules in Metasploit. We
+ already have script classes which run once per port and once per
+ host. For example, the once-per-scan ("network script"?) class might
+ be useful for broadcast LAN scripts (Ron Bowes, who suggested this
+ (http://seclists.org/nmap-dev/2010/q1/883) offered to write a
+ NetBIOS and DHCP broadcast script). Another idea would be an AS to
+ IP ranges script, as discussed in this thread
+ http://seclists.org/nmap-dev/2010/q2/101 [Could be a good SoC
+ infrastructure project]
+ o David notes: "I regret saying this before I say it, because I'm
+ imagining implementation difficulties, we should think about
+ having such auxiliary scripts be able to do things like host
+ discovery, and then let the following phases work on the list it
+ discovers."
+
+o Analyze what sort of work would likely be required for Nmap to
+ support OS detection over IPv6 to a target.
+ o Would probably start with a way to send raw IPv6 packets
+ o There is a raw IPv6 patch here:
+ http://seclists.org/nmap-dev/2008/q1/458
+ o Also it looks like Nping may be doing this already.
+ o Then we need to figure out if we can use our current DB and
+ techniques, or if we'd likely thave to have an IPv6-specific
+ DB. [David]
+
+o July Nmap releases (at least a beta version, and maybe a stable
+ too). Last release was 5.30BETA1 on March 29
+
+o Add this patch for compilation on OpenSolaris.
+ http://blogs.sun.com/sdaven/entry/nmap_5_35dc1_compile_on
+
+o Now that we've put the ndiff, ncat, and nping man pages under the
+ scope of the book (e.g. https://nmap.org/book/ncat-man.html), we need
+ to add a redirect from the old locations and also update our links.
+
+o Make sure the long output lines in Nping's man page are OK for the book.
+ See r18829 and r18864.
+
+o Update "History and Future of Nmap"
+ (https://nmap.org/book/history-future.html) to include all the news
+ since September 2008. [Fyodor]
+
+o Fix Win7 networking issue reported by Luis which seems to have been
+ triggered by r17542. See this thread:
+ http://seclists.org/nmap-dev/2010/q3/40
+
+o Upgrade to WinPcap 4.1.2 - Rob has a patch - See this thread:
+ http://seclists.org/nmap-dev/2010/q3/18
+
+o [NSE] Review UnrealIRCd backdoor detection script
+ http://seclists.org/nmap-dev/2010/q2/854
+
+o [Zenmap] Investigate segfault on some installs of OS X 10.6.3:
+ http://seclists.org/nmap-dev/2010/q2/587
+ o David rebuilt with MacPorts 1.9.1 rather than 1.8.2 and the
+ problem went away.
+
+o [Zenmap] Investigate failure to start on some installations of OS X
+ 10.6.3.
+ [ We think one may just not have waited long enough as he said it
+ started working, and another case (the 587) seems to be a
+ segfault--we added a new task for that ]
+ http://seclists.org/nmap-dev/2010/q2/587
+ http://seclists.org/nmap-dev/2010/q2/859 (He responded to David
+ privately and said that it was not an I7 processor.)
+ Nmap seems to be having problems too:
+ http://seclists.org/nmap-dev/2010/q2/747
+
+o [NSE] Review Gutek's PHP version disclosure script.
+ http://seclists.org/nmap-dev/2010/q2/569
+
+o Fix the IPv6 name resolution problem described in this thread:
+ http://seclists.org/nmap-dev/2010/q2/787
+
+o [NSE] Review Gutek's libopie detection/DOS script.
+ http://seclists.org/nmap-dev/2010/q2/635
+
+o [NSE] Review Gutek's web server directory traversal script.
+ http://seclists.org/nmap-dev/2010/q2/595
+ - It became modifications to http-passwd
+
+o [NSE] Review dns-cache-snoop.nse from Eugene Alexeev.
+ http://seclists.org/nmap-dev/2010/q2/195
+ Better attachment at: http://seclists.org/nmap-dev/2010/q2/200
+ Need to decide on a domain list: http://seclists.org/nmap-dev/2010/q2/199
+
+o Fix bug where multiple targets with the same IP can end up in a
+ hostgroup and cause port scanning and probably OS detection to
+ misbehave. An example is "nmap -F scanme2.nmap.org
+ scanme3.nmap.org". See this thread for details:
+ http://seclists.org/nmap-dev/2010/q2/322
+
+o Need to fix our current win32.zip distribution so that .svn files
+ aren't included (currently they are in nselib/data). Will probably
+ be a simple adjustment to mswin32/Makefile.
+
+o Make Zenmap splash screen
+
+o [NSE] Add one of, or combine, ntp-peers and ntp-monlist.
+ http://seclists.org/nmap-dev/2010/q2/190
+ http://seclists.org/nmap-dev/2010/q2/191
+
+o [NSE] Reorganize nselib to allow libraries in subdirectories.
+ Currently, to avoid expanding the number top-level libraries, code
+ that is only used by one library is built into that library's file,
+ even if it is logically separate. For example, the mongodb library
+ contains a BSON-parsing library. Instead, that library could go in
+ mongodb/bson.lua. The msrpc and smb libraries could potentially be
+ broken up in this way.
+ UPDATE: We decided not to do this for now, given complications in
+ nsedoc, packaging, etc. to support the new hierarchy. Instead, we
+ can use prefixes like we do with scripts (e.g. mongodb-bson.lua,
+ msrpc-types.lua).
+
+o Add a configure option to our libpcap which enables an older Linux
+ packet capture system (David's noring patch). This is needed in
+ some cases for 32-bit static binaries to work on 64-bit Linux
+ systems. Note that it is unneccessary if both the build system and
+ the target system use Linux 2.6.27, as that has an architecture
+ independent tpacket_hdr (called tpacket2_hdr). [Added by David as
+ --disable-packet-ring]
+
+o Test Jay Fink's UDP payload prototype.
+ http://seclists.org/nmap-dev/2010/q1/168
+ [ tested, improved, merged by David]
+
+o Resolve Ncat broadcast support issue (see this thread:
+ http://seclists.org/nmap-dev/2010/q2/422).
+
+o [NSE] Review and test the DB2 library and
+ scripts. http://seclists.org/nmap-dev/2010/q2/395 (but updated
+ versions may be available).
+
+o Move nmap/docs/TODO into its own todo directory (probably nmap/todo)
+ and then encourage maintainers of /status/ TODOs and any other TODOs
+ to migrate theirs there. Unlike the status directory, /nmap/todo
+ would be readible by anyone. [Fyodor]
+
+o Nmap should at least print (and maybe scan) all IP addresses for
+ hostnames specified on the command line. We will start with just
+ printing all the addresses. Here is a thread on the topic:
+ http://seclists.org/nmap-dev/2010/q2/302
+ [David made it do the printing, adding a different task related to
+ scanning them all]
+
+o Integrate new service detection fingerprint submissions (we have
+ more than 730 since Dec. 17, 2009.
+
+o [Ncrack] Use our new password lists (now used by NSE) for Ncrack as
+ well. Ncrack can probably handle a larger list than NSE uses.
+
+o Consider MSRPC ideas from Ron--we might want to add some as TODO
+ tasks: http://seclists.org/nmap-dev/2010/q2/389
+
+o Fix XML inconsistency described at
+ http://seclists.org/nmap-dev/2010/q2/326
+
+o Integrate new OS fingerprints (we have more than 1,300 since
+ November 10, 2009).
+
+o Finish selecting GSoC 2010 projects
+
+o Upgrade libpcap to the new 1.1.1 version.
+
+o Improve the NSI installer by adding command-line options for unsetting
+ each of these GUI checkboxes individually (particularly useful for
+ silent mode):
+ LangString DESC_SecCore ${LANG_ENGLISH} "Installs Nmap executable, NSE scripts and Visual C++ 2008 runtime components"
+ LangString DESC_SecRegisterPath ${LANG_ENGLISH} "Registers Nmap path to System path so you can execute it from any directory"
+ LangString DESC_SecWinPcap ${LANG_ENGLISH} "Installs WinPcap 4.1 (required for most Nmap scans unless it is already installed)"
+ LangString DESC_SecPerfRegistryMods ${LANG_ENGLISH} "Modifies Windows registry values to improve TCP connect scan performance. Recommended."
+ LangString DESC_SecZenmap ${LANG_ENGLISH} "Installs Zenmap, the official Nmap graphical user interface. Recommended."
+ LangString DESC_SecNcat ${LANG_ENGLISH} "Installs Ncat, Nmap's Netcat replacement."
+ LangString DESC_SecNdiff ${LANG_ENGLISH} "Installs Ndiff, a tool for comparing Nmap XML files."
+ LangString DESC_SecNping ${LANG_ENGLISH} "Installs Nping, a packet generation tool."
+
+o We should have a standard function which takes time arguments in the
+ same format as Nmap does (e.g. 60s, 1m, etc.) and the scripts which
+ take time arguments should be modified to use it. David suggests
+ this here: http://seclists.org/nmap-dev/2010/q2/35. We are also
+ going to update the normal Nmap timing functions to take seconds by
+ default, as described here: http://seclists.org/nmap-dev/2010/q2/159
+
+o Nmap should probably always produce a well-formed XML file, even if
+ it exits with a fatal() error. In that case, the error should be
+ included in the XML. Right now, for example, if the network is
+ down, the XML output will just stop (no closing tags) and Nmap will
+ print something to STDERR like:
+ nexthost: failed to determine route to 9.48.184.164
+ QUITTING!
+
+o Get @output sections for the last remaining scripts w/o them:
+ [WARN] script auth-spoof missing @output
+ [WARN] script db2-das-info missing @output
+ [WARN] script db2-info missing @output
+ [WARN] script http-passwd missing @output
+ [WARN] script iax2-version missing @output
+ [WARN] script ms-sql-config missing @output
+ [WARN] script ms-sql-query missing @output
+ [WARN] script oracle-sid-brute missing @output
+ [WARN] script pop3-brute missing @output
+ [WARN] script pptp-version missing @output
+ [WARN] script skypev2-version missing @output
+
+o [Zenmap] Maybe it should sort IPs in an octet-aware way. And maybe
+ you should be able to sort by IP address (perhaps that should be the
+ default). Current plan is to just sort by IP by default, and maybe
+ we'll offer other sort techniques later if desired. See
+ http://seclists.org/nmap-dev/2010/q2/27 [possible SoC student task]
+
+o Brainstorm for GSoC 2010 ideas and fill out the org application by
+ Friday 3/12 4PM PST.
+ o NSE scripts
+ o Maybe a whole SoC role for http scripts
+ o Maybe look at other web app scanners for some inspiration
+ (including w3af - http://w3af.sourceforge.net/)
+ o Maybe a non-http developer too
+ o NSE infrastructure manager
+ o Ncrack
+ o Nping
+ o Mobile Devices? N900, iPhone, Android
+ o Zenmap developer
+ o Must have solid user interface design experience
+ o Zenmap script selector (subset of a Zenmap or NSE SoC role)
+ o Feature Creepers/Bug fixers
+
+o Review IDS detection scripts from Joao Correa.
+ http://seclists.org/nmap-dev/2010/q1/814
+
+o Review mssql library and scripts from Patrik Karlsson.
+ http://seclists.org/nmap-dev/2010/q1/1000 (files)
+ http://seclists.org/nmap-dev/2010/q1/1014 (sample output)
+
+o Review DNS fuzzer script from Michael Pattrick.
+ http://seclists.org/nmap-dev/2010/q1/1005
+
+o Our nsedoc generator should probably give a warning if a script is
+ missing any important fields. @output comes to mind. @usage can be
+ nice too, though we could consider auto-generating that for trivial
+ scripts.
+
+o [NSE] Consider pros and cons of splitting information retrieval
+ scripts into a bunch of small single-purpose script vs. one larger
+ argument-controlled script. See
+ http://seclists.org/nmap-dev/2010/q1/1023
+ [we ended up combining three of the ms-sql scripts. If we combine
+ future scripts, we need to remember to add them to the deprecation
+ list in the Makefile]
+
+o Remove --interactive. It was broken for a long time and nobody
+ seemed to notice, and we put a call out on nmap-dev for
+ --interactive users and didn't get any good reasons to keep it. We
+ should kill it to remove the code complexity it adds and to avoid
+ the documentation complexity of people having to read and learn
+ about a feature they are unlikely to ever use.
+
+o Zenmanp should perhaps be able to print Nmap output on a Printer (if
+ not too much of a pain to implement.)
+
+o Review afp-serverinfo.nse from Andrew Orr.
+ http://seclists.org/nmap-dev/2010/q1/470 Just waiting on some bug fixes:
+ http://seclists.org/nmap-dev/2010/q1/665
+
+o Test 64-bit pcap installer (e.g. remove old version and install new)
+ before next release, as we've applied a change from Rob which works on
+ his system (http://seclists.org/nmap-dev/2010/q1/796).
+
+o [NSE] Improve username/password library (the database files
+ themselves). We don't have very good lists at the moment. Maybe
+ work in combination with Ncrack dev.
+ o Now there are some even better lists available (f.e. RockYou)--see
+ this thread: http://seclists.org/nmap-dev/2010/q1/764
+ o We've improved the ncrack files--we should probably either use
+ those for NSE or use a subset of them.
+ o perhaps from Solar Designer. (he sent us permission)
+ o perhaps add phpbb hack data (there is at least a list of 28,635
+ passwords in phpbb_users.sql, and possibly more in other files.
+
+o [Nping] Should take the version number 0.[nmap version], such as
+ 0.5.22TEST
+
+o Review rpc.lua, nfs-showmount.nse, nfs-get-stats.nse, and
+ nfs-get-dirlist.nse from Patrik Karlsson.
+ http://seclists.org/nmap-dev/2010/q1/270
+
+o [NSE] Look into moving packet module to C for better performance
+ [Patrick]
+ o Removing this one because it is stale (has been here for many
+ months with no action seen), but it is something we can consider
+ if/when there is a desire to implement it. A key is probably to
+ measure current performance and see if it is a material problem.
+
+o Maybe the Nmap ASCII art should come after make rather than
+ configure?
+ - We decided it would probably be annoying for developers to see it
+ every time they 'make'.
+
+o Review snmpenum.nse from William Njuguna.
+ http://seclists.org/nmap-dev/2009/q4/721
+ http://seclists.org/nmap-dev/2010/q1/656
+ o Dropping for now unless original author or someone else picks it
+ up and fixes the bugs.
+
+o Add smtp-enum-users from Duarte Silva if testing is favorable.
+ http://seclists.org/nmap-dev/2010/q1/699
+
+o After the new -sn and -Pn options (added to SVN around 7/20, just
+ after the 5.00 release) have been around long enough to be in most
+ people's copy of Nmap (e.g. in all the versions we distribute from
+ download page (stable+dev)) for at least a few months, we'll document
+ these as the preferred version rather than -sP and -PN. These match
+ -n, and the main problem with -sP is that we now use it more for
+ "disable portscan" than ping only. For example, you can also use
+ NSE, traceroute, etc. [David]
+
+o Nmap currently selects routes based on the first matching one it
+ finds. But it should really take the most specific route instead.
+ So it should:
+ 1) Keep searching the routing table for the most specific match, and
+ 2) Use a stable sort (not qsort) so that routes with identical
+ netmasks aren't rearranged.
+ For more, see http://seclists.org/nmap-dev/2010/q1/685
+
+o Review pgsql-brute.nse from Patrik Karlsson.
+ http://seclists.org/nmap-dev/2010/q1/455
+
+o psexec missing (need to download yourself now) nmap_services.exe
+ output issue: "The function where this is detected returns a value
+ that is passed to stdnse.format_output. format_output takes a
+ parameter to decide whether it's displaying an error message, but it
+ is hard-coded to only display error messages with debugging >= 1. So
+ options are to change format_output and make it more flexible, or
+ somehow decouple the sensing of nmap_service.exe from the normal
+ output channel of the script."
+
+o Website: Create shared directory in svn, which will contain
+ directories shared between the Insecure.org network of sites
+ (e.g. templates, error, css). Then sites such as sectools,
+ nmap.org, insecure.org can just check that out via externals
+ declaration (or, I suppose, symlink). CSS directives will then use
+ /shared/css/insecdb.css etc. ).
+
+o Add CouchDB and JSON scripts once the JSON library is finished.
+ http://seclists.org/nmap-dev/2010/q1/641
+
+o Review NSE raw IP from Kris Katterjohn.
+ http://seclists.org/nmap-dev/2010/q1/559
+
+o Review sslv3-enum.nse from Mak Kolybabi.
+ http://seclists.org/nmap-dev/2010/q1/563
+
+o [NSE] Consider LDAP library and scripts from Patrik Karlsson.
+ http://seclists.org/nmap-dev/2010/q1/70 [all merged, except David is
+ still reviewing ldap-search]
+
+o More potential improvements to http-methods:
+ http://seclists.org/nmap-dev/2010/q1/630 and
+ http://seclists.org/nmap-dev/2010/q1/640
+
+o Remove smtp-open-relay.nse sometime after 9/24/09 if nobody adopts it (see
+ http://seclists.org/nmap-dev/2009/q3/0986.html). [It got fixed up
+ and we kept it.]
+
+o The -v and -d arguments should take the same syntax. Right now you
+ use -vvv vs. -d3. We should probably just make either approach work
+ with either of them.
+
+o Zenmap should be able to export normal Nmap output
+
+o Integrate Nping.
+
+o [NSE] Consider the http-methods script from Bernd Stroessenreuther.
+ http://seclists.org/nmap-dev/2010/q1/76. [integrated, but David is
+ making some improvements].
+
+o The Nmap web page is beginning to show its age. Ah, who am I
+ kidding, it was showing its age 5 years ago :). It could do with an
+ upgrade to XHTML+CSS. It could also do with a whole redesign, but I
+ think that can be done as a second step after converting to
+ XHTML+CSS with roughly the same look. Though adding a few more
+ modern touches (like hover interaction on the menu bar) wouldn't
+ hurt. This is a moderatly big project, which will involve: o
+ Designing the new XHTML+CSS to look similar to the current HTML
+ pages, but be extensible enough that it can be redesigned in the
+ (near) future by mostly just changing the CSS and graphics.
+ o Converting the existing Nmap pages to the new XHTML format.
+ This will likely include using open source programs and likely
+ modifying them or creating your own scripts to help with the
+ process. To apply for this task, you need to have some web
+ development experience and an example XHTML+CSS web page you
+ have created online.
+ o We decided not to worry about XHTML for now, and we're
+ integrating CSS in piece by piece -- we already have the section
+ headers, left sidebar links. etc.
+ o Should not use SSI like the current pages -- should do all its
+ magic through CSS. That way it will work on seclists too (which
+ can't do SSI for security reasons).
+ o Maybe alpha transparency for menus, gradiants, curves, etc. But
+ the main goal isn't flashiness.
+
+o Seclists.org should maybe be fixed so that it doesn't strip quoted
+ text for its summaries from the IP list because that list consists
+ almost entirely of forwarded material which is being stripped. Look
+ at the summaries at http://seclists.org/interesting-people/.
+
+o Web site HTML improvements
+ - Maybe start with nmap.org.
+ - Find and fix HTML validation problems, bad links. I'm not sure
+ what tool is best for this.
+ - Then do the same with seclists.org, insecure.org, sectools.org
+ - The icon on the top-left of the screen should be for (and link
+ to) the root URL of current site. e.g. seclists.org,
+ sectools.org, nmap.org rather than always insecure.org.
+
+o [NSE] Consider SNMP scripts from Patrik Karlsson.
+ http://seclists.org/nmap-dev/2010/q1/162
+ http://seclists.org/nmap-dev/2010/q1/174
+ http://seclists.org/nmap-dev/2010/q1/178
+
+o Deal with AV false positive issue RE nmap_services.exe:
+ - For now, David is going to apply Ron's patch which removes this,
+ but David will make it print output in verbose mode rather than
+ debug and maybe make it a little less verbose. LT plan is for Ron
+ to encrypt it with OpenSSL.
+
+o Web site improvements
+ - Update to use CSS, at least for header bars
+ - Also, if it is easy to give the header bars rounded corners,
+ we should probably do so. But if it is hard, it isn't
+ important enough to matter.
+ - The Nmap.Org navigation table should have a background and more
+ subtle lines, like we use for our calendars now.
+ - The first item (table) in featured news has slightly more
+ left/right margin than the later ones on Firefox 3.5.6, and with
+ IE8 it doesn't extend as far when you make the page really wide.
+ Plus the images on the right are problematic (extend through the
+ border below them) when you make the window too wide on IE8.
+ Having a slight margin on the left/right of entries would
+ actually be a bit nice. And it would be nice if it only took a
+ simple tag or two, controlled by CSS rather than pasting in a
+ whole table with font tags and the like for each entry.
+
+o [Ncat] Test, review, and (if appropriate) merge Venkat's HTTP Digest
+ proxy authentication patch. See
+ http://seclists.org/nmap-dev/2009/q3/773. [David]
+
+o [NSE] Look at new DB2 script by Tom
+ Sellers. http://seclists.org/nmap-dev/2009/q4/659
+
+o [NSE] Consider MongoDB scripts and libraries from Martin Holst Swende.
+ http://seclists.org/nmap-dev/2010/q1/177
+
+o [NSE] Document Patrick's worker thread patch in scripting.xml (see
+ http://seclists.org/nmap-dev/2009/q4/294,
+ https://nmap.org/nsedoc/lib/stdnse.html#new_thread,
+ https://nmap.org/nsedoc/lib/nmap.html#condvar) [Patrick]
+
+o Make Nmap 5.21 bugfix-only release
+
+o [NSE] Consider afp-showmount script from Patrik Karlsson.
+ http://seclists.org/nmap-dev/2010/q1/97
+ [merged to trunk]
+
+o [NSE] Review DNS-SD script from Patrik Karlsson.
+ http://seclists.org/nmap-dev/2010/q1/87
+ [merged to trunk]
+
+o [NSE] Consider MySQL scripts from Patrik Karlsson.
+ http://seclists.org/nmap-dev/2010/q1/163
+ [merged to trunk]
+
+o [NSE] Consider DAAP script from Patrik Karlsson.
+ http://seclists.org/nmap-dev/2010/q1/164
+ [merged to trunk]
+
+o NSEDoc left sidebar should include a link to
+ https://nmap.org/book/nse.html below "Index".
+
+o Consider enhancing the new OS Assist system to handle version
+ detection too. [We decided not to do this as David noted that Doug's
+ serviceunwrap.lisp does pretty much everything he needs.]
+
+o [NSE] HTTP header parsing is not very robust, and is duplicated in a
+ lot of places. For example, it's legal to have header fields like
+Content-type:\r\n
+___text/html\r\n
+(with spaces in place of _, but http.lua won't parse such a header
+correctly. In other words you can extend them to any number of lines
+as long as each line after the first begins with whitespace. [David]
+
+o Investigate issue with our Pcap and Wireshark x64, as described in
+ this thread: http://seclists.org/nmap-dev/2009/q4/557 [Rob]
+ [Taking this off the list until/unless we get more reports]
+
+o Decide what to do about Windows 7/Vista and starting NPF. See this
+ thread: http://seclists.org/nmap-dev/2010/q1/20
+
+o [NSE] We should do a favicon survey like the one Brandon did for
+ /favicon.ico files but which uses the favicons specified by the HTML
+ files rather than just that exact location. For example, insecure.org
+ sites include in the headers:
+ <link REL="SHORTCUT ICON" HREF="http://images.insecure.org/images/tiny-eyeicon.png" TYPE="image/png">
+ Then we should update our favicon database to include the top ones,
+ and we should also improve our favicon script so that it either
+ omits checking /favicon.ico if the HTML-specified one exists, or it
+ should just download, interpret, and display info for both (right
+ now it seems to give prority to the wrong one: /favicon.ico).
+
+
+o [Ncat] Add SSL support for --exec so you can use SSL to talk to your
+ remote shell, etc. See this thread:
+ http://seclists.org/nmap-dev/2009/q4/255, particularly the
+ implementation sketch at http://seclists.org/nmap-dev/2009/q4/268 [Venkat,David]
+
+o Look at new Kerberos script from Patrik Karlsson.
+ http://seclists.org/nmap-dev/2009/q4/715 . [We decided not to merge
+ this one since its usefulness turned out to be limited on Windows and
+ very limited on any other platform. ]
+
+o Add feature to http library to let user set the user agent to be
+ used. The NSEDoc for this feature should probably tell what our
+ current default user agent is ("Mozilla/5.0 (compatible; Nmap
+ Scripting Engine; https://nmap.org/book/nse.html") [David]
+
+o On our NSEDoc pages (e.g. https://nmap.org/nsedoc/), perhaps the link
+ text for scripts should not include the ".nse". Basides saving
+ horizontal space, this may improve the sorting so that the likes of
+ "citrix-enum-apps" comes before "citrix-enum-apps-xml". Also, we can
+ probably get away with reducing the width of the NSEDoc left-column,
+ especially if ".nse" is removed.
+
+o [NSE] Patrick's script dependency patch:
+ http://seclists.org/nmap-dev/2009/q4/295
+ o I'm not sure if he has gone through and actually set appropriate
+ dependencies (and removed runlevels) yet
+
+o Integrate latest version detection submissions and corrections.
+ This was last done based on submissions until February 9, 2009.
+
+o Release 5.10BETA2
+
+o Add --evil to set the RFC3514 evil bit.
+ ftp://ftp.rfc-editor.org/in-notes/rfc3514.txt
+ o We're not going to add this right now.
+
+o Talk to Libpcap folks about incorporating (at least some of) my
+ changes from libpcap/NMAP_MODIFICATIONS. [marking as done since the
+ upstream-appropriate changes are pretty minor now that we've
+ upgraded to 1.0]
+
+o Nping -- like hping3 but uses Nmap infrastructure and to a
+ large degree the same command-line options as Nmap.
+ [We now have an alpha version at https://nmap.org/nping/]
+
+o Further investigate SCTP functionality, as some people reported
+ problems (see this thread:
+ http://seclists.org/nmap-dev/2009/q2/0669.html)
+
+o [NSE] NFS query script for checking exports, etc.? [Patrik Karlsson]
+
+o [NSE] Attempt to reproduce and fix a deadlock reported by Brandon
+ when he does large-scale scanning with a new favicon script with
+ hostgroups as small as 8,192 (he hasn't seen it with 4096
+ hostgroups). Could be a bug in internal NSE socket lock. Probably
+ not specific to the favicon script, but that is how Brandon
+ reproduces it. At the hang, stack trace is usually the threads stuck
+ in socket_lock function, sometimes lookup_cache mutex in http
+ library. David guesses that it's threads being garbage-collected
+ from the socket lock table. The only thing that can wake up a thread
+ waiting on a socket lock is if a thread that holds a lock is removed
+ from the table. But the table has weak keys, meaning that a thread
+ can be garbage collected and it will be automatically removed from
+ the table by the Lua runtime. Then there is no event that can wake
+ up a thread waiting for a lock. [David and Patrick made some commits
+ at end of November meant to resolve this, and we haven't seen the
+ problem since, so we're marking it as done for now].
+
+o Look into reducing Nmap memory consumption
+ o UDP scans with -p- and large hostgroups are a particularly large
+ offender. See if there is a way to prevent them from eating up
+ gigs of RAM. See the message "Port memory bloat" at
+ http://seclists.org/nmap-dev/2009/q3/0926.html for a patch that
+ reduces Port memory use by about 50%.
+ o One idea David has been considering is a way to represent filtered
+ ports (or whatever the default state is) without creating a Port
+ object for each one.
+ [David]
+
+o Fix assertion failure with certain --exclude arguments (see
+ http://seclists.org/nmap-dev/2009/q4/276). [David]
+
+o Many people may have stale (since removed/renamed) scripts in their
+ Nmap scripts directory because our 'make install' does not remove
+ them and so they remain and can cause problems (like running twice
+ after being renamed). We should probably add a line to our 'make
+ install' which removes the scripts/lib names we have previously
+ used. We're doing this rather than blowing away the old directory
+ just in case someone has custom scripts/libs there (though that is
+ still a bad idea). [David]
+
+o Update the CHANGELOG for new 5.10BETA1
+ release. [Fyodor]
+
+o Make the new Nmap 5.10BETA1 release
+
+o Ndiff man page should be built from XML source whenever a release is
+ done, as ncat/zenmap/nmap man pages are. [Fyodor]
+
+o We should package the rendered Nroff man page translations (e.g. all
+ 16 languages) in the tarball to make it easier for distributors to
+ package them. For example, see
+ http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=358336. Including
+ the translations would add 2.5MB to the (currently 28MB)
+ uncompressed tarball and about 800KB to the (currently 9MB) bz2
+ compressed tarball. [Fyodor]
+
+o The Nmap 5.00 tarball contains:
+ -rw-r--r-- fyodor/fyodor 122943 2009-06-24 14:35 nmap-5.00/docs/scripting.xml
+ -rw-r--r-- fyodor/fyodor 151 2009-06-24 14:35 nmap-5.00/docs/nmap-usage.xml
+ -rw-r--r-- fyodor/fyodor 604 2009-06-24 14:35 nmap-5.00/docs/nmap-man-enclosure.xml
+ -rw-r--r-- fyodor/fyodor 76918 2009-06-24 14:35 nmap-5.00/docs/nmap-install.xml
+ -rw-r--r-- fyodor/fyodor 10179 2009-06-24 14:35 nmap-5.00/docs/legal-notices.xml
+ If we're going to include the XML source files, we should include
+ refguide too. But rather than add that, we should probably take
+ these out. After all, people can easily grab them from svn or our
+ new http svn gateway if desired. So no need to bloat the tarball
+ with these files which aren't installed. [We're going to take the
+ XML source files out of the tarball] [Fyodor]
+
+o Consider converting this file to emacs org-mode
+ (http://orgmode.org/) format. [Fyodor]
+ o That format is still plain text and can be read/edited by vi
+ users, etc.
+ [Considered, but I don't think I'll change right now]
+
+o Windows 7 RTM Nmap testing (With particular attention to 64-bit and
+ our pcap installer). [Fyodor]
+
+o We should print host latency (when available) in the XML output, as
+ suggested at http://seclists.org/nmap-dev/2009/q4/215.
+ docs/nmap.dtd will have to be modified accordingly, and you might
+ even consider adding support to docs/nmap.xsl.
+
+o Integrate latest OS fingerprint submissions and corrections. This
+ was last done based on submissions up to May 8, 2009.
+
+o Potential OS X 10.6 problems. There are two issues reported by the
+ same user which may be related:
+ http://seclists.org/nmap-dev/2009/q3/0936.html,
+ http://seclists.org/nmap-dev/2009/q3/0996.html. One is that Nmap
+ hangs doing nothing and needs to be killed with Ctrl-C, and the
+ other is that it dies after printing "Initiating UDP Scan". Another
+ reported the same problem at
+ http://seclists.org/nmap-dev/2009/q3/0990.html, where it dies after
+ the first ARP request is sent. But Brandon has run Nmap on 10.6
+ without problems. It is a bit of a mystery. [David] [Resolution:
+ Apple fixed the problems in 10.6.2; For users who have 10.6 and
+ 10.6.1, the versions David builds on 10.5 will still work for them
+ because they are 32-bit binaries rather than 64. Users who build
+ Nmap on 10.6 or 10.6.1 should compile with -m32 or update to 10.6.2]
+
+o [NSE] Patrick's worker thread patch:
+ http://seclists.org/nmap-dev/2009/q4/294
+
+o Investigate get_rpc_results error (infinite loop) reported by Lionel
+ Cons. See these threads: http://seclists.org/nmap-dev/2009/q4/24,
+ http://seclists.org/nmap-dev/2009/q4/120
+
+o Upgrade to latest version of NSIS on Nmap Win build system [Fyodor].
+
+o Standardize on a proper file header for the Zenmap source code. [David]
+ o For now, David is going to augment the templatereplacement system
+ to insert the normal nmap.header.tmpl, but change the comment format
+ to work with Python, and then replace the current Zenmap headers
+ with that.
+
+o We may want to look into if/how we support IPv6 nameservers. Here
+ is a bug report from someone having a problem with them:
+ http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=539244 [Ankur]
+
+o Once all the man page languages are in the Nmap tarball, we should
+ update our install system to install them in the appropriate place.
+ We'll want to integrate this with configure so users can decide which
+ languages they want. See http://seclists.org/nmap-dev/2009/q4/249.
+
+o Resolve allow_ipid_match issue which can cause some malformed
+ replies to be ignored when we might be able to still use them. See
+ this thread: http://seclists.org/nmap-dev/2009/q2/665 [David]
+
+o Fix Zenmap 'make install' TypeError issue
+ (http://seclists.org/nmap-dev/2009/q4/225). [David]
+
+o Fix a bug in which Nmap can wrongly associate responses to SYN and
+ ACK host discovery probes. [David]
+ For example:
+ # nmap -sP -PS80 -PA80 australia.gov.au --packet-trace -d2
+ SENT (0.0760s) TCP 192.168.0.21:60182 > 152.91.126.70:80 S ttl=43 id=13466 iplen=44 seq=4046449223 win=4096 <mss 1460>
+ SENT (0.0770s) TCP 192.168.0.21:60182 > 152.91.126.70:80 A ttl=48 id=39976 iplen=40 seq=4046449223 win=1024 ack=921915001
+ RCVD (0.3020s) TCP 152.91.126.70:80 > 192.168.0.21:60182 SA ttl=53 id=0 iplen=44 seq=3924706636 win=5840 ack=4046449224 <mss 1380>
+ We got a TCP ping packet back from 152.91.126.70 port 80 (trynum = 0)
+ ultrascan_host_probe_update called for machine 152.91.126.70 state UNKNOWN -> HOST_UP (trynum 0 time: 226875) Changing ping technique for 152.91.126.70 to tcp to port 80; flags: A
+ In the example above, Nmap wrongly uses ACK as the preferred ping technique, when it should be SYN. [David]
+ o we're thinking about ways to encode the information better. Right
+ now we have pingseq and tryno, but we may want to just move to a
+ single probe ID and then we can look up any other information in
+ structures attached to that ID in memory when we get the response.
+ o A related problem, which we hope the fix for this will also
+ resolve, is that replies can currently match any probe whose tryno
+ is less than or equal to the tryno encoded in the reply.
+ o However, "fixing" this problem has been shown in the past to
+ cause accuracy problems. See
+ http://seclists.org/nmap-dev/2009/q1/387. We should figure out
+ whether we can still reproduce that and, if so, what is going on
+ before "fixing" this issue.
+
+o Add PJL (Printer Job Language) probes to
+ nmap-service-probes. Brandon wrote some in
+ http://seclists.org/nmap-dev/2009/q1/0560.html. Test them to see if
+ they cause anything to be printed out (on paper) with printers that
+ don't support PJL. If not, then remove the JetDirect ports from the
+ default exclude list. The script pjl-ready-message.nse also uses
+ PJL. We have concerns about the safety of this probe given
+ http://seclists.org/nmap-dev/2009/q4/61, but it still is probably
+ better to have the probe in there than not, as long as we continue
+ blocking the ports by default with the Exclude directive.
+ [We put in the probes, but are keeping the Exclude directives
+ because the probes still seem a bit dangerous]
+
+o [NSE] in_chksum in packet.lua doesn't work with an odd number of
+ bytes. Also make it more efficient.
+
+o Add --confdir option to Zenmap. See
+ http://seclists.org/nmap-dev/2009/q1/92 [David]
+
+o Update our Winpcap from 4.0.2 to 4.1.1
+ (http://seclists.org/nmap-dev/2009/q4/128). This is a bit complex
+ because we have our own installer. See
+ https://nmap.org/svn/mswin32/winpcap/Upgrading-Instructions.txt.
+
+o Change Nmap to not show the "Host not scanned" lines in list scan
+
+o Change Nmap to show latency in "host is up" lines even w/o verbose
+ mode.
+
+o Update our included Libpcap from 0.9.7 to 1.0.0
+ (http://www.tcpdump.org/) [David]
+
+o Improve Nmap output to show the forward DNS name when specified on
+ command line as well as rDNS where appropriate. We're also going to
+ reorganize output to enable some other improvements as well. See
+ the proposal at http://seclists.org/nmap-dev/2009/q3/814, and that
+ whole thread which starts at
+ http://seclists.org/nmap-dev/2009/q3/805 [David].
+
+o [Zenmap] Solve some unusual utf8 Zenmap crashes reported in the
+ crash reporter. David has fixed some of them so far, but there are a
+ few more remaining that may be related. [David]
+
+o Change Nsock to give an error if you try to FD_SET a fd larger than
+ FD_SETSIZE. [Brandon]
+ o Some research from David:
+ We have help off on this change because of Windows portability
+ problems. The Windows fd_set works differently than the Unix
+ fd_set. In Unix, FD_SETSIZE (which is typically 1024) is both the
+ maximum number of file descriptors that can be in the set and one
+ greater than the greatest file descriptor number that can be
+ set. In other words, we want to bail out whenever someone tries
+ to FD_SET file descriptor 1060, for example. But on Windows it's
+ different: FD_SETSIZE is only 64, but any file descriptor
+ numbers, no matter how great, may be stored in the set. Windows
+ socket descriptors are typically greater than 1023, but you can
+ only have 64 of them in the set at once.
+
+ So the fix on Unix would be
+ --- nsock/src/nsock_core.c (revision 15214)
+ +++ nsock/src/nsock_core.c (working copy)
+ @@ -97,6 +97,7 @@
+ do { \
+ assert((count) >= 0); \
+ (count)++; \
+ + assert((sd) < FD_SETSIZE); \
+ FD_SET((sd), (fdset)); \
+ (max_sd) = MAX((max_sd), (sd)); \
+ return 1; \
+ @@ -107,6 +108,7 @@
+ assert((count) > 0); \
+ (count)--; \
+ if ((count) == 0) { \
+ + assert((sd) < FD_SETSIZE); \
+ FD_CLR((sd), (fdset)); \
+ assert((iod)->events_pending > 0); \
+ if ((iod)->events_pending == 1 && (max_sd) == (sd)) \
+
+ But that doesn't work on Windows (I just tried it) because even
+ the smallest socket descriptor is bigger than FD_SETSIZE, 64.
+ Really we're trying to accomplish two different things on the two
+ platforms: On Unix we must not store a file descriptor greater
+ than 1023, no matter how many or how few other descriptors have
+ been set. On Windows we must not set more than 64 descriptors at
+ a time, no matter what their descriptor number happens to be.
+
+o Add a way in NSE to set socket source addresses and port numbers.
+ See this thread: http://seclists.org/nmap-dev/2009/q3/821. Some
+ potential solutions are discussed later in the thread.
+
+o [Ncat] Fix --max-conns on Windows so that it only counts concurrent
+ connections and not long-dead ones. See this thread
+ (http://seclists.org/nmap-dev/2009/q3/1017.html) and particularly this
+ message (http://seclists.org/nmap-dev/2009/q3/1032.html) for
+ details. Venkat has a patch for David to review and potentially merge.
+
+o [Ncat] Fix 100% CPU usage with ncat -l --send-only. See this
+ thread: http://seclists.org/nmap-dev/2009/q2/797 and continues
+ further at http://seclists.org/nmap-dev/2009/q3/99. This message is
+ key: http://seclists.org/nmap-dev/2009/q3/308 [David]
+
+o [Seclists] There is currently some extra vertical space after the
+ first post of a thread in the thread index (example:
+ http://seclists.org/nmap-dev/2009/q4/index.html).
+
+o [NSE] Decide which scripts belong to the "safe" category (we now have 20
+ which aren't either safe or intrusive), then remove the intrusive
+ category since people can now specify "not safe". See
+ http://seclists.org/nmap-dev/2009/q3/1091.html and that whole
+ thread. [Fyodor]
+ [ OK, see http://seclists.org/nmap-dev/2009/q4/0002.html]
+
+o [NSE] Fix http pipelining. Responses are being split on anything
+ that looks like HTTP/1.X which doesn't come at the beginning of a
+ line, and doesn't work when a line like that happens to legitimately
+ come in a body. Joao has an nmap-exp branch which resolves this
+ issue, though David found some bugs in that and sent some hard test
+ cases. [Joao]
+
+o Fix traceroute performance/algorithms. It is terribly bad in some
+ cases. For example, this traceroute scan took 36 minutes against a
+ single host(!): http://seclists.org/nmap-dev/2009/q3/0425.html . We
+ don't need to go up to hop 50 in such cases (maybe some heuristic
+ like "at least go to hop 15, and stop after 5 unresolved in a row).
+ And more importantly, there is no reason each hop should take 40s to
+ timeout. It should probably use timeout variables like we use in
+ port scanning. And it should parallelize as much as possible. Even
+ if parallel resolution means we went a little further than we had to
+ in incrementing the TTL, and we go to hop 15 when host is at 12
+ that's no big deal (of course we would only report up to hop 12 in
+ the output). Once we do this, we should put back the ability to
+ make --traceroute work even when we haven't found a probe which
+ elicits a response from the target. (that feature was added in July,
+ but we'll probably take it out until we can fix
+ performance). [David]
+
+o Fix four Nmap bugs discovered by Ankur and analyzed a bit by
+ David. [Ankur]
+
+o [NSE] Consider HTTP request caching.
+
+o [NSE] Finish (or write new) favicon fingerprinting script. See
+ http://seclists.org/nmap-dev/2008/q4/0583.html . May need to do
+ some more scanning and increase the DB size a bit. May or may not
+ want to later combine this as part of a larger webapp fingerprinting
+ script.
+
+o [Zenmap] When the inventory is changed, the current host/service selection is
+ forgotten and the Ports / Hosts tab is switched to hosts mode. It should
+ remember your current selection and not change the view. [David/SoC]
+
+o Device categorization improvements
+ o Examine Nmap's device categorization in nmap-os-deb and
+ nmap-service-probes. Decide if some small categories which have
+ never really took off should be consolidated, or whether others
+ should be split off. For example, maybe there are some groups in
+ 'specialized' or other misc. categories which are now large enough
+ to split off. Personally, I wouldn't give anything its own
+ category unless there are at least half a dozen of them and no
+ other category really fits them well. We should use a combined
+ system for nmap-os-db and nmap-service-probes.
+ o Add a classification sect1 to os-detection.xml
+ (https://nmap.org/book/osdetect.html) to cover how Nmap handles OS
+ classification. It should include a list with descriptions of
+ each device type recognized by Nmap. Version-detection.xml should
+ reference (link to) it in the approprate place.
+ [Doug has done some initial work on this. For example, see
+ nmap/docs/device-types.txt] [David]
+
+o Consider what new UDP payloads we might want to add. David has many
+ ideas at: http://seclists.org/nmap-dev/2009/q3/0290.html
+
+o For traceroute we should give some indication that the RTT is in ms.
+ Changing the column header to maybe "RTT MS" or "RTT (MS)" would
+ probably do the trick or we could append "ms" to each value.
+ [David]
+
+o OS fingerprint should probably specify somewhow when DS=1 if it's
+ because target->directlyConnected is true, or because it sent the
+ distance probe and calculated a distance of 1. The second situation
+ should never happen, but often David strongly suspects that it is the
+ case.
+
+o --traceroute should probably set currenths->distance because right
+ now, I do an -O scan against scanme.nmap.org, and it does not figure
+ out the distance. So the fingerprint shows no distance element and
+ Nmap doesn't print "Network Distance" in the results line. That may
+ be OK (Nmap probably isn't receiving the probe response needed for
+ this, and maybe doesn't want to print the TG), but even when I do
+ --traceroute I get no distance printed. Yet Nmap clearly knows the
+ distance since the traceroute shows all the hops up to and including
+ the target (scanme.nmap.org).
+
+o Figure out best favicon to use for Nmap and related web sites
+ [David]
+
+o [Ncat] David says: "After you get EOF on stdin with --send-only, the
+ program hangs on until the idle timeout expires instead of terminating
+ immediately. I had a fix for it but it involved deleting events in
+ the Nsock queue and it caused an assertion failure in Nmap so I backed
+ it out. I have a less intrusive solution." [David]
+
+o We should update our config.{sub,guess} files. This Debian bug
+ #542079 requests that we do so:
+ http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=542079. We last
+ updated on 3/15/08 and in that case we used versions from
+ http://cvs.savannah.gnu.org/viewvc/config/?root=config. That may or
+ may not be the best place to get them now (e.g. perhaps there has
+ been a recent official release). [David]
+
+o Look a bit more at default version detection timing. Particularly
+ deciding the number of probes to run in parallel. [ We increased
+ that a bit on 8/18/09]
+
+o [Ncat] Right now our -i (idle timeout) causes Ncat to quit if EITHER
+ reading or writing is idle for the given amount of time. But it is
+ really only idle if BOTH reading AND writing are idle for the
+ period. We should make the code work that way.
+
+o Add scripting.xml documentation on strict.lua and the avoidance of
+ global vars in libraries. See
+ http://seclists.org/nmap-dev/2009/q3/0169.html. Probably a new
+ section just above "Adding C Modules to "Nselib", such as "Writing
+ Your Own Library" or somesuch. [Patrick]
+
+o Update nsedoc to refer to 'libraries' rather than 'modules'. This
+ affects the front page (which calls them 'Libraries' on left sidebar
+ and 'Modules' on the list of right, and affects the url (we should
+ change /modules/ to /lib/ and then have Fyodor add a redirect for
+ people still using old URLs) and the title of the module pages like
+ https://nmap.org/nsedoc/modules/base64.html. [Patrick]
+
+o [Ncat] Prefix Ncat stderr messages with "Ncat: " to make it clear
+ that they are coming from Ncat and not the remote server (or typed in
+ by user). [David/SoC]
+
+o [NSE] Optimize NSE Performance--e.g. measure the current performance and
+ see what can be improved in terms of scheduling scan threads,
+ determining how many to run concurrently, looking at CPU load items,
+ etc. [David/Patrick]
+
+o Increase version scan concurrency based on Patrick's performance
+ testing. We decided to go to 20 for timing_level 3, 30 for 4, and 50
+ for 5.
+
+o [NSE] Consider POST/HEAD support. See
+ http://seclists.org/nmap-dev/2009/q1/0889.html.
+ o Implemented: http://seclists.org/nmap-dev/2009/q3/0074.html
+ o Joao going to check in very soon soon.
+
+o [NSE] Consider Rob Nicholls http-enum script for incorporation:
+ http://seclists.org/nmap-dev/2009/q1/0889.html
+ [Joao tested w/his HEAD support, is going to check this in]
+
+o Consider the open proxy scripts more carefully
+ - How should we test whether the proxy attempt was successful? Right
+ now we look for a google-specific Server header after trying to
+ reach http://www.google.com through the proxy. Maybe we should let
+ users specify their own pattern if they specify their own URL.
+ [ Joao is going to check it in today (7/28)]
+
+o I should add code to Nmap to bail if sizeof(char) isn't 1.
+ Otherwise there could be security risks if it is not one on any
+ platforms. [ Actually, we think C standard requires this and we've
+ not heard of any system where sizeof(char) isn't 1. So removing
+ this item.]
+
+o [Zenmap] More complete implementation of ZenmapCommandLine/profile
+ editor improvement ideas. See
+ http://www.bamsoftware.com/wiki/Nmap/ZenmapCommandLine. [David]
+
+o [Ncat] Think about whether we should offer "-q secs" (quit after EOF
+ + delay of secs) and/or -k (set SO_KEEPALIVE on socket) (or maybe
+ that should be set by default). Anyway, these were suggested here:
+ http://lwn.net/Articles/341706/ [We're going to fix -i (added
+ separate item), and not worry about SO_KEEPALIVE unless we see more
+ demand for it. It doesn't seem that nc110 or OpenBSD nc or so-called
+ GNU Netcat support SO_KEEPALIVE either]
+
+o [Ncat] In verbose mode, I'd like to see clock time (duration) and
+ maybe in/out traffic stats when a client connection ends. Maybe it
+ could use a format similar to what Nmap provides. [David/Venkat]
+
+o Seriously consider making --traceroute work even when we haven't
+ found a probe which elicits a response from the target. We'd just
+ have to pick a probe in that case (probably echo request, as we
+ found that to be the most effective in prev. empirical testing).
+ This is similar to UNIX traceroute and Windows tracert.exe which
+ just pick a probe (high UDP port on UNIX, ICMP echo request on Win).
+ Even if the host is down or something, we usually get some useful
+ hop information.
+
+o [NSE] Allow spaces in script arguments without the user having to
+ manually quote them (beyond normal shell escape quoting). See:
+ http://seclists.org/nmap-dev/2009/q3/0090.html
+ [Patrick]
+
+o [Ncat] Support SCTP now that Nmap does.
+ - See client support patch by Daniel Roethlisberger:
+ http://seclists.org/nmap-dev/2009/q2/0609.html
+ - Server support?
+ - Daniel has a patch, David looking to apply once an nsock thing is fixed.
+
+o Look at etc/payloads.conf in unicornscan-0.4.7 and see if they have
+ any which we don't have, but should, for our version detection.
+ They have a decent collection there. KX sent some other programs we
+ should look at too. [David]
+
+o Ncat should give it's ethernet cat ASCII logo after
+ configure--similar to the way that Nmap, Ncrack, and Nping
+ do. [David/SoC]
+
+o [Zenmap] The Search dialogue is helpful for finding a certain scan
+ you've performed recently, but we should probably also offer a similar
+ function for searching for certain applications/hosts within a scan
+ (e.g. find all the hosts running Apache). This new functionality
+ might be a find option or some other mechanism rather than being
+ part of the Search dialogue proper.
+
+o Ncat SSLv2 issues. See
+ http://seclists.org/nmap-dev/2009/q1/0319.html. A big part of it is
+ done, which was enhanced version detection probes to detect more SSL
+ servers, The defect that remains is that Nsock can't connect to a
+ small fraction of servers (including some of the ones detected by
+ the new version probe). They are the servers that do only SSLv3 or
+ TLSv1 and don't respond to a SSLv2-compatible ClientHello. Even
+ though most servers don't support SSLv2, they usually respond to the
+ ClientHello and just don't offer any SSLv2 features. [David/Venkat
+ working on this]
+
+o Deadlock identification and correction:
+ o Plan of action: implement freeing of script mutexes when scripts
+ exit without freeing them (done and in /nmap now). And then if it
+ continues to be a problem we'll consider this other stuff:
+ o Add detection for deadlocks and print which threads are involved.
+ o use above results to make a strategy for automatic deadlock resolution.
+ o Original entry: Figure out what to do about NSE mutexes:
+ http://seclists.org/nmap-dev/2008/q3/0276.html . In particular, they
+ are not currently cleaned up if a thread dies or otherwise exits
+ without unlocking them and can cause endless deadlocks which are
+ annoying to users and can be difficult to debug :(. Patrick has
+ some ideas for this in his SoC09 proposal:
+ "Adding a cleanup system for NSE that is called periodically
+ similar to nsock_loop. There would be a registration system
+ allowing C libraries to register a Lua function that will run
+ periodically to check for irresolvable deadlock or simply dead
+ resources. For example, the nmap library would register a mutex
+ cleanup handler which would inspect all mutexes looking for a dead
+ thread or circular dependencies. The nsock library could register
+ a handler that checks for unused sockets. The nsock may save a
+ strong reference to the thread that owns the socket and inspect it
+ to determine if the thread is dead."
+ David later says: "After some discussion we decided to start more
+ modestly, first by ensuring that a scripts mutexes are released when
+ it dies for whatever reason. I have a hunch that this is the cause
+ of most deadlocks. It was certainly the cause of two whois.nse
+ deadlocks I found. Then, the next step if deadlocks continue to be a
+ problem, is to do automatic detection and just print out a list of
+ what scripts are involved. It could be that several smb scripts are
+ deadlocked, or as in the case I observed where whois.nse was locked
+ with itself."
+
+o Joao is auditing his Lua code to make sure all his variables are
+ local where appropriate. [Joao - done, should be commited very soon]
+
+o [NSE] We need to deal with libraries which improperly use global
+ variables, as that is very common (Patrick made a list:
+ http://batbytes.com/bad.txt). Solutions could involve augmenting
+ our runtime system (the "strict.lua" approach) to detect/prevent the
+ problem, a script we run occasionally to identify issues that we
+ then manually resolve, or, at the very minimum, documenting
+ somewhere in scripting.xml the dangers inherent in global variables
+ and warn people to generally declare them local instead. We have a
+ long history of bugs caused by non-local variables defined in NSE
+ libraies and often causing deadlocks.
+
+o The Nmap refguide (https://nmap.org/book/man-performance.html) says
+ "The --max-parallelism option is sometimes set to one to prevent Nmap
+ from sending more than one probe at a time to hosts. This can be
+ useful in combination with --scan-delay (discussed later), although
+ the latter usually serves the purpose well enough by itself." But
+ when you actually try it:
+ # ./nmap --max-parallelism 1 --scan-delay 10 scanme.nmap.org
+ You can't use --max-parallelism with --scan-delay.
+ QUITTING!
+ We need to either make that work or adjust the documentation. [David/SoC]
+ o David changed this to a warning. Note that with --scan-dealy,
+ --max-parallelism is essentially 1 anyway.
+
+o [NSE] Consider integrating HP Laserjet print PJL status-setting
+ script. See this thread for an example of such a script:
+ http://seclists.org/nmap-dev/2009/q3/0083.html (note that it is
+ updated during the thread). Also, see this thread:
+ http://seclists.org/nmap-dev/2009/q3/0092.html
+
+o Ndiff man page should be expanded to include sample execution/output
+ and more fully describe its functionality. [David]
+
+o David is going to reexamine the old coverity-reported issues (the
+ ones we previously marked as "ignore" because they weren't real bugs)
+ just to be sure that is (and is still) the case.
+
+o Make -sP work with -PN to disable both port and ping scanning. We
+ need to make sure the various options still work (-O, --script,
+ --traceroute, etc.) with this, as many currently don't as they don't
+ expect this behavior, which used to be unsupported and cause Nmap to
+ quit with an error messaqge. It may be OK to refuse -O since that
+ will rarely give useful results. OTOH, -O may work on some systems
+ with unique closed port signatures where Nmap guesses a closed
+ port. Users should then be able to do an NSE-only scan with "-sP -PN
+ --script [scripts]" We should document this -sP -PN usage in
+ refguide. [David]
+
+o Add -sn and -Pn options which are aliases for -sP and -PN. Once
+ they've been around long enough to be in most people's copy of Nmap,
+ we plan to document those as the preferred version. Those match -n,
+ and the main problem with -sP is that we now use it more for
+ "disable portscan" than ping only. For example, you still might
+ want to use NSE. [David]
+
+o [NSE] Make sure all our HTTP scripts transparently support SSL
+ servers too. [Joao has a solution and is testing the http scripts to
+ make sure they don't break.]
+
+o Resolve "memcpy overlap in getinterfaces(int*) (tcpip.cc:2987)".
+ See this thread: http://seclists.org/nmap-dev/2009/q2/0713.html
+ [David/Brandon]
+
+o [Ncat] Print a message to stderr upon connection failure even if -v
+ isn't specified so the user knows what went wrong. [David/SoC]
+
+o [Ncat] Maybe --chat should imply -l. And Maybe --broker should too?
+ - OTOH, we might want to extend --chat for connect mode in the
+ future.
+ [We're going to hold off on chat now, David/SoC is doing --broker]
+
+o Consider making it easier to tell whether scripts were specified by
+ name on the command-line (rather than default or by class) so they
+ have the option of providing extra verbosity in that case. For
+ example, see http://seclists.org/nmap-dev/2009/q2/0563.html. We
+ could either provide a special function for scripts to determine
+ that, or we could magically adjust nmap.verbosity() when called by
+ those scripts. [David]
+
+o [NSE] Figure out a way to support people who want to do script scan,
+ but not port scan or ping scan. One option would be to allow
+ --script to list scan (-sL), but perhaps a better option is to
+ provide a way to disable port scanning in the same way as we offer
+ -PN to disable ping scanning. As an example of this need, David had
+ to write special code to avoid ping/port scanning when doing a
+ whois.nse survey for
+ http://www.bamsoftware.com/wiki/Nmap/EffectivenessOfPingProbes. The
+ key for this task is to figure out how to do it from a user
+ interface perspective and then implement and document it. We've
+ already been going in the direction of allowing script scanning in
+ more types of scans--a while back we started allowing it with -sP
+ ping scans due to high demand. [David/SoC]
+ [ We decided how we're going to do it (-sP -PN to start out with;
+ leading to eventual -sn -Pn) and added new TODO entries for actually
+ doing the code/docs. ]
+
+o Ndiff should be able to show NSE script result changes. [David]
+
+o Get set up for Coverity scan of latest version to see if it catches
+ any important issues before stable release. [Fyodor,David]
+ [Found 7 new results, 3 are real bugs, and 2 have been fixed so far]
+
+o [nsock] Fix Makefile to handle dependencies correctly (if that turns
+ out to be the problem). See
+ http://seclists.org/nmap-dev/2009/q1/0629.html. o Or it may be
+ related to SVN timestampling. See
+ http://seclists.org/nmap-dev/2009/q1/0632.html. Diagnosed by David:
+ http://seclists.org/nmap-dev/2009/q2/0728.html
+
+o For at least our UDP ping probes, Nmap should probably notice if it
+ is a very well known service port such as 53, 161, or 137 and send
+ an appropriate probe packet (server status for DNS, public community
+ string query for SNMP, etc) rather than empty data in that case.
+ This is similar to the way our IP protocol probes automatically
+ include common headers such as TCP and UDP if that common protocol
+ is given. Good probes for these services are already available in
+ nmap-service-probes, though we might want to make a custom file for
+ this. We should probably do this for port scanning as well. [David]
+
+o [NSE] Make NSE work better for SSL tunneled services in general by
+ supporting them easily in the libraries. For example, I don't think
+ irc-info.nse currently works against all the servers which tunnel
+ over SSL. Maybe augment comm library, etc. [Joao - done, except for
+ http, which is already a separate TODO item]
+
+o Update scripts which use table args to use pseudo-table format
+ "name.arg" rather than requiring the user to create a Lua table
+ themselves. On the lua side, it's not really being stored in a
+ table, but just an arg named "name.arg". [Joao]
+ - Look at all our existing scripts which use tables
+ (dns-zone-transfer, whois, the proxy scripts, etc.) and change as
+ appropriate. Remember to change the usage throughout the script
+ and also change the nsedoc script arguments and example usage.
+ For the existing scripts, try to retain the table version check
+ for now to avoid breaing backward compatability if possible. Just
+ add the newer style check as well.
+ - Is taking arguments in a table specific to a script a good idea?
+ The example in the socks-open-proxy nsedoc of "--script-args
+ openproxy={host=<host>}" is a bit of a mess and I'm not sure the
+ best way to document that in the script argument list. Note that
+ this is the standard way we've handled it for some other scripts,
+ so it's not an open-proxy-script-specific problem.
+
+o [NSE] Track active sockets in the nsock library binding and don't
+ rely on garbage collection for reallocation. Can probably wait until
+ post-stable release for integration. [Patrick]
+ - Patrick has a patch and is waiting on dev branch to check it in.
+
+o [NSE] Resolve ssh2.lua buffering problems
+ (http://seclists.org/nmap-dev/2009/q2/0673.html) [Joao]
+
+o Decide what to do about ncat source code headers -- maybe just use
+ the Nmap ones. [David added the Nmap headers]
+
+o Once we go into deep stability freeze mode, create an nmap-exp
+ development branches for changes we plan to integrate after the
+ stable release. [Fyodor]
+
+o Update CHANGELOG for latest changes [Fyodor]
+
+o Release 4.85BETA10
+
+o [NSE] Open proxy detection scripts
+ o We have http-open-proxy.nse, but we should probably either extrand
+ that to handle other types of proxies (such as SOCKS and HTTP
+ CONNECT) or create more scripts to handle those other proxy
+ types. [Joao, David]
+ o Joao has written scripts, just need to finish up, evaluate, integrate.
+
+o Determine whether zenmap.spec.in can currently require
+ "python-sqlite" rather than "python-sqlite2", or if it at least can
+ be easily made to do so. The former seems more compatible since
+ RHEL/CentOS 5.3 has a "python-sqlite" package, but not
+ "python-sqlite2". Meanwhile, Fedora 10 provides the "python-sqlite"
+ capability as long as you have the Python 2.5 package installed
+ (python-2.5.2-1.fc10). Fedora 10 does also make a
+ python-sqlite2 package available.
+
+o [Ncat] Solve EOF issues which crop up when piping to an external
+ command. See http://seclists.org/nmap-dev/2009/q2/0528.html. It
+ sounds like we will go with Daniel's patch [Daniel, David]
+
+o Look into building RPMs with SSL support. Statically linking to
+ OpenSSL on Linux for the RPMs didn't work for me last time I
+ tried. [Fyodor]
+ o Static linking of Nmap to OpenSSL does not seem to work on Fedora
+ 10 or CentOS 5.3. The problem appears to relate to the OpenSSL
+ krb5 support.
+ o Could build my own OpenSSL libraries on the build system
+ (w/o Kerberos support) and link to those.
+ o At some point, we might want to consider including OpenSSL with
+ Nmap tarball. The problem is that it is rather big. Would
+ increase Nmap .tar.bz2 size from about 9 megs to about 12. OTOH,
+ OpenSSL is only going to get more and more important. Maybe we
+ can include a stripped down version?
+ o If we don't integrate OpenSSL (or until we do), we might consider
+ a more prominent configure warning for when SSL is not detected.
+ We could suggest that users run "yum install libopenssl-devel" or
+ "apt-get install libssl-dev" commands or whatever is appropriate
+ and then reconfigure. Or we could point them to a page or
+ nmap-dev posting URL with instructions.
+
+o Figure out why I [Fyodor] get a bunch of "Operation not permitted" errors
+when I launch a scan on SYN such as:
+ - I'm going to ignore this for now unless it causes me trouble
+ again, as this is an old machine that will be replaced soon anyway.
+ And we haven't been hearing of the problems from others lately.
+ /home/fyodor/nmap-exp/fyodor-perf/nmap -nogcc -T4 -n -v -p- --portpingfreq 250 -oA /home/fyodor/nmap-misc/logs/WorldScan/portpingfreq/logs/portpingfreq-250-1%T-%D 67.15.236.34 67.15.236.36 81.174.236.66 81.174.236.119 170.140.20.160 170.140.20.174 202.138.180.9 202.138.180.17 202.138.180.132 209.20.64.112
+ The errors look like:
+sendto in send_ip_packet: sendto(7, packet, 44, 0, 170.140.20.174, 16) => Operation not permitted
+Offending packet: TCP 64.13.134.4:59820 > 170.140.20.174:59120 S ttl=39 id=19927 iplen=44 seq=2425535549 win=4096 <mss 1460>
+sendto in send_ip_packet: sendto(7, packet, 44, 0, 67.15.236.36, 16) => Operation not permitted
+Offending packet: TCP 64.13.134.4:59820 > 67.15.236.36:15030 S ttl=57 id=50640 iplen=44 seq=2425535549 win=2048 <mss 1460>
+Discovered open port 49394/tcp on 170.140.20.174
+sendto in send_ip_packet: sendto(7, packet, 44, 0, 170.140.20.174, 16) => Operation not permitted
+Offending packet: TCP 64.13.134.4:59819 > 170.140.20.174:8256 S ttl=48 id=38510 iplen=44 seq=2425601084 win=1024 <mss 1460>
+ May be related to connection tracking and high scan rates. See
+ http://seclists.org/nmap-dev/2008/q4/0652.html
+ http://www.shorewall.net/FAQ.htm#faq26
+ Others have reported similar issues even without connection tracking. See
+ http://seclists.org/nmap-dev/2006/q3/0277.html
+ http://seclists.org/nmap-dev/2007/q2/0292.html
+
+
+o -PO1 and "-sO -p1" seem to send ICMP ping packets with an ICMP ID
+ field of 0, which we found that a small percentage of hosts drop
+ (61.13% responded with 0, 62% with a random value). So we might as
+ well randomize them in these cases. [Josh Marlow]
+
+o Some of the -PS443 scans (and maybe other ones) we've been running
+ have been missing the Nmap line telling how many packets were
+ sent/received, even though we had verbose mode. [David/Josh]
+
+o Deal with Ncat newline problem. See this thread:
+ http://seclists.org/nmap-dev/2009/q2/0325.html [David,Jah]
+
+o Integrate SCTP scanning support. See Daniel Roethlisberger's branch
+ in nmap-exp/daniel/nmap-sctp. As of 4/30/09, he is nearing
+ completion. See http://seclists.org/nmap-dev/2009/q2/0270.html.
+
+o [NSE] Release mutexes upon script death to prevent certain deadlocks
+ [Patrick, David]
+
+o Consider whether to let Zenmap Topology graph export the images to
+ svg/png/etc. Also think about printing. Note that João Medeiros
+ has written a Umit patch to do this: [Joao, David]
+ http://trac.umitproject.org/ticket/316.
+ - Now he has Nmap patch:
+ http://seclists.org/nmap-dev/2009/q2/0409.html
+ - Consider integrating.
+ - Integrated!
+
+o Ensure that when I build a distribution package on UNIX (e.g. make
+ distro), it builds what is in the Nmap directory I am calling it
+ from rather than a particular SVN version. I'm going to start
+ building packages from a special "clean" directory which is
+ different than the one I do development work in. Also, I want to be
+ sure that any changes in that dir are included in the release, even
+ if they aren't check in yet. [Fyodor]
+
+o Nmap UNIX distro build script should regenerate script.db. [Fyodor]
+ o Now it is in make prerelease
+
+o Nmap build system should be split into [Fyodor]
+ o prerelease -> generates version files, man pages, script.db
+ etc. That has to be done on one system, and then results checked in
+ before doing a make release. It does this stuff based on the
+ directory it is run in rather than some set dirname or a pure SVN
+ version
+ o release-tarballs -> does any system-dependent building and creates
+ the source tarballs. It does this stuff based on the directory it
+ is run in rather than some set dirname or a pure SVN version
+ o release-rpms -> Same as above, but also uses the created tarballs
+ to build the Linux RPM binaries for the current platform based on the
+ tarballs.
+
+o Build x86 and x86-64 VM instances for RPM building. [Fyodor]
+ * I think I'll use CentOS 5.3
+
+o [NSE] Script scanning does not seem to work on Fyodor's Linux
+ machines after being installed from latest SVN (or 4.85BETA9) and run
+ as a non-root user (it works fine as root). The command "nmap -sC
+ localhost" leads to NSE failure messages which differ based on the
+ exact version run. [Was a relatively simple permissions problem in
+ our Makefile.in -- I fixed it]
+
+o [NSE] Release socket locks on connection failure or
+ timeout. [Patrick]
+
+o Update Nmap entry on Linux Online -
+ http://www.linux.org/apps/AppId_1979.html
+ - Screw it, the site does not seem to be maintained at all. They
+ aren't taking updates as of 6/2/09, and even Firefox shows latest
+ update as 0.9.1.
+
+o [Ncat] In verbose mode, print when an SSL connection is established
+ successfully and give the leaf certificate hash to make it easier to
+ verify when connecting to a machine where you can't or don't want to
+ use --ssl-verify (e.g. connecting to an ncat ssl server where it
+ created its own key). While we're at it, we might want to print
+ some other information from the leaf node, such as organizationName
+ and maybe localityName, countryName or something. We don't want to
+ be too verbose, but 1 line would be great and 2-3 might be
+ acceptable. [David]
+
+o Fix NSEdoc to better escape single-quotes in fields. If we can't do
+ that for some reason, we need to document it better. For example,
+ when we initially tried generating nsedoc for
+ http-webdav-unicode-bypass.nse, NSEdoc was listing it as a module
+ named "s auxiliary module", apparently because this line exited in
+ the description field:
+ This module is based on Metasplit's auxiliary module, modules/auxiliary/scanner/http/wmap_dir_webdav_unicode_bypass.rb.
+ (For full example, see scripts/http-webdav-unicode-bypass.nse
+ r13345) [David/SoC]
+
+o --script-args should allow a wider range of characters, and should
+ give a more useful error message if it receives chars it really
+ can't handle for some reason. For an example, try
+ "--script-args=smbuser=admin,smbpass=pass^word". For more details,
+ see Ron's report at
+ http://seclists.org/nmap-dev/2009/q2/0378.html.
+
+o [Ncat] Have --ssl-cert and --ssl-key send a certificate in connect
+ mode so that client certificate auth can be done. [David/Venkat]
+
+o Once we're done with host discovery empirical research, add it to
+ host-discovery.xml. Would be great to show the best combinations to
+ use for a given number of probes, the efficiency of the common probes
+ by themselves, etc.
+
+o Consider making the ping scan default be more comprehensive. Note
+ that I got 23% more Internet boxes found out of a 50K sample (see host
+ enumeration chapter of my book for details). Maybe I should
+ experiment a bit more to ensure they are real boxes and not network
+ artifacts and figure out exactly which tests are helping the most.
+ If I do this change, I'll have to update the host enumeration
+ chapter. For UDP probing purposes, we should test whether including
+ extra data in the packet (e.g. --data-length) helps in general, and
+ for services such as 53 and 137, we should probably send proper
+ protocol headers (e.g. a DNS server status message) so that we
+ receive responses from listening services.
+
+o We should probably check for a system Lua in a "lua5.1" directory
+ rather than just "lua", as Debian and also my Fedora 10 systems seem
+ to have that. See
+ http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=527997. [Note,
+ Fyodor asked the bug reporter Jan Nordholz on 5/14/09 if he could
+ write a patch. Jan sent in a patch, it worked, Fyodor checked it in.]
+
+o [NSE] Get rid of ceil so that floating point NSE runlevels work
+ again (some scripts, including (smb-brute) rely on this. They got
+ broken with the NSE core lua rewrite. [David].
+
+o NSE script logical operator stuff is now documented in
+ scripting.xml--add to refguide.xml as well. [David/Patrick]
+
+o [NSE] Correct nsock_connect to unlock the socket slot if the
+ connection fails. When a socket is closed, it is unlocked so the
+ arbitrator can potentially open up a socket for another thread. But
+ Patrick discovered that a socket is not automatically unlocked when
+ a connection fails or times out, only when it is closed
+ explicitly. So that could hold up socket allocation for other
+ threads until garbage collection. May be a cause of slowness or
+ possibly deadlocks. [Patrick]
+
+o [NSE] Solve segfault issue which occurs when Nsock events call back
+ on a thread that has already ended (e.g. timeout, crash, early exit,
+ whatever) and been garbage collected. May want to just nsi_delete
+ all nsock sockets immediately upon thread ending. For an example of
+ this type of segfault, see
+ http://seclists.org/nmap-dev/2009/q2/0289.html. David says " I think
+ in the interests of getting this in a stable release, we should use
+ that strategy of closing all a thread's sockets. That ought to fix
+ all the problems above. Not to rule out a more thoughtful redesign
+ in the future." [David,Patrick]
+
+o We added the SEQ.CI value in Feb 2009 with 0 matchpoints. At some
+ point (once we have some real-life values) we need to evaluate whether
+ we want to give it points. A good time to do that would be when we
+ next do fingerprint integration, so we will actually have examples
+ of .CI in the nmap-os-db. [David]
+
+o [NSE] Make it a warning rather than error if a script in script.db
+ can't be found. [Patrick]
+
+o Add version detection signature for Ncat chat once we finalize the
+ announce format. [David]
+
+o Change Nmap signature files to use the .sig extension rather than
+ .gpg.txt, as that seems to be what gpg recommends. In fact, gpg
+ will automatically verify the right file if it exists after dropping
+ the .sig (or .asc) extension. I may need to configure .htaccess to
+ serve .sig files properly. Update nmap-install.xml
+ accordingly. Suggested by tic at eternalrealm.net by email on
+ 7/13/08. [Fyodor]
+ * Rename existing files, add symlink from the old .gpg.txt to .asc
+ versions
+ * Add appropriate .htaccess content type if needed for downloads
+ - not needed since I decided on .asc extension rather than .sig
+ * Update the generation scripts
+ * Update the book documentation -
+ https://nmap.org/book/install.html#inst-integrity
+
+o Ask Coverity if they'll scan latest version of Nmap. [Fyodor asked
+ David Maxwell on 5/14/09 ]
+
+o Make 4.85BETA9 release [Fyodor]
+
+o [Zenmap] Make a way to start a scan from the profile editor without
+ creating a profile, then remove the command wizard. This is partial
+ implementation of
+ http://www.bamsoftware.com/wiki/Nmap/ZenmapCommandLine. [David]
+
+o [Ncat] Make proxy server mode work on Windows (this is the last
+ remaining fork() dependency in Ncat).
+
+o Do an OS detection integration run -- last was based on
+ 1/8/09. [David]
+
+o [Ncat] Maybe we should create an SSL cert with no passphrase during
+ Ncat compilation or install process so that if someone specifies
+ Ncat -l and --ssl with no --ssl-cert and --ssl-key, we already have
+ one for them, and it is a slightly better one (since the private key
+ isn't known) than if we distributed a key. Obviously it is still
+ subject to MITM attacks since there is no domain validation going
+ on. But people who need that will have to buy a key from a
+ certificate authority in any case. We could create the key by using
+ the "openssl" command line tool as shown in
+ https://nmap.org/ncat/guide/ncat-advanced.html#ncat-ssl, or maybe
+ better to have a way for ncat to do it using openssl calls. [David]
+
+o [Zenmap] Should probably give some sort of widget indication that a
+ scan is running. Now that we can start multiple scans at once, the
+ "scan" button goes back to being unpressed while the scan is
+ running. As some scans take minutes or more to show output, it is
+ not always clear whether they are still properly running. We should
+ probably have some sort of widget, such as the throbber used in web
+ browsers, to show that Nmap is still running. It could be fore a
+ specific scan (kind of like how you have a separate throbber for
+ each tab on a web browser), or a global one which means at least one
+ scan is running. Or maybe a different sort of indication is in
+ order (like a timer). [David]
+
+o Further investigate Nmap Proxy patch by Zoltan Panczel and Ferenc
+ Spala. See http://nmap-dev.fw.hu/ and
+ http://seclists.org/nmap-dev/2009/q1/0255.html . [Discussed it and
+ then added new proxy feature item]
+
+o Wherever practical, fix compiler warnings when compiling Nmap with
+ VC++ 2008 Express SP1 (there aren't many). [David]
+
+o [NSE] Consider adding boolean expressions to --script arguments. For
+ example, see Patrick's implementation at
+ http://seclists.org/nmap-dev/2008/q3/0300.html .
+
+o Generate a list of trusted SSL certificates to ship with Ncat (by
+ extracting f rom Mozilla or similar), and install them with
+ Ncat. Decide how these certificat es should be preferred to any
+ system-provided certs, if any. [David]
+
+o [NSE] Add desired SoC09 infrastructure ideas to this TODO to the
+ extent they don't already exist.
+
+o [Ncat] Consider supporting server certificate verification when used
+ in client SSL mode.
+ o For now we document in user's guide that it is not secure.
+ o Maybe we can do an ssh-style approach where we just print the
+ fingerprint and expect the ncat client user to ensure it is the
+ right one?
+ o If we're going to verify cert's etc., we need to also make sure we
+ are actually using secure ciphers. We may need to update nsock to
+ support cipher selection, because we want fast ones for version
+ detection, but usually want secure ones for NSE and/or ncat.
+ o Do we want to check all this by default, or offer an option for
+ it? Doing it by default is more secure, though it can be annoying
+ when a certificate has expired, is self-signed, you connect to
+ domain.com when the certificate is for www.domain.com, etc. If it
+ is done by deault, we might just print an error message. Whreas
+ if we have a special option, it may be OK to exit and refuse the
+ connection.
+ o What certs should we allow? Same as the browsers do? Maybe get
+ rid of Comodo? Maybe we should fail to recognize any certs with MD5
+ in the trust chain?
+ o What about people who are running their own SSL service and just
+ want to specify the cert file they use, because they generated it
+ themself and not from a trusted CA.
+ o Need to check expiration, domain, etc. if we're checking certs at
+ all.
+ o We can probably get away with not doing revocation checking, as
+ long as we document that we don't.
+
+o consider changing status field from "up" and "down" to "online" and
+ "offline". Actually, maybe we don't want this after all.
+ online/offline look pretty similar, and they're longer too. I'm
+ taking this out of the TODO.
+
+o [Ncat] When acting as an HTTP proxy, we should support GET mode as
+ well as CONNECT so that it works as a non-SSL proxy in browsers such
+ as firefox. [David]
+
+o Finalize GSoC applicant research, communication, and selection
+ [David, Fyodor]
+
+o Go through all the SoC applicants and decide who we want to accept
+ and start communicating with them. [David,Fyodor]
+ o Decide which applicants we want, and who would be best for
+ mentoring them.
+
+o Document that U1.RID gives "G" as long as all the data bytes in the
+ echoed response data are "C" as expected. This G code is still
+ given even when the response is truncated, including if there are 0
+ bytes echoed. [David]
+
+o [Ndiff] Rethink the output format. David says: In particular, I
+ would like to always have the old state on the left and the new
+ state on the right: "was filtered, is open," not "is open, was
+ filtered." I also like the context diff output of MadHat's
+ nmap-diff. [David]
+
+
+o Canonicalize the "host up" messages for port scan and ping scan so
+ that instead of things like "Host scanme.nmap.org (64.13.134.52)
+ appears to be up ... good." we standardize in both cases on
+ something like: "Host scanme.nmap.org (64.13.134.52) is up (.75s
+ latency)". Note the addition of the latency value, which is our
+ srtt value for the host. This will only show in ping scan and
+ verbose port scan because the line doesn't appear without verbose
+ mode. [David]
+
+o Ping scans always seem to say "0 [hosts] undergoing Ping Scan" when
+ you request stats, rather than the proper number. For an example,
+ try a command such as "nmap -iR 10000 -sP -n" and then press enter
+ during the scan. Here are some examples of the bad output: Stats:
+ 25:34:33 elapsed; 991232 hosts completed (72530 up), 0 undergoing
+ Ping Scan Ping Scan Timing: About 53.69% done; ETC: 22:49 (0:00:09
+ remaining) Stats: 0:01:10 elapsed; 0 hosts completed (0 up), 0
+ undergoing Ping Scan Ping Scan Timing: About 24.03% done; ETC: 22:42
+ (0:03:41 remaining) Stats: 0:03:28 elapsed; 4096 hosts completed
+ (284 up), 0 undergoing Ping Scan Ping Scan Timing: About 3.06% done;
+ ETC: 22:44 (0:03:07 remaining) [David]
+
+
+o Remove obsolete tests from nmap-os-db itself. [David]
+
+o Prepare for Summer of Code
+ * Brainstorm for ideas
+ * Create new ideas page
+ * Apply to participate in program again
+ * Advertise for applicants
+ * Evaluate applicants
+
+o NSEDoc script/module documentation pages should probably provide a
+ link to the script/module source code (except for C modules). The
+ link format should probably be of the form
+ https://nmap.org/data/scripts/[script].nse and
+ /data/nselib/[module].lua. NSEdoc can assume they already exist
+ there, as we'll probably put them there using the same system we use
+ to copy other stuff to the data dir.
+
+o [Ncat] Let people set up authenticated proxies using
+ --listen and --proxy-auth together (right now we don't support
+ that). [David]
+
+o When you specify multiple comma-separated arguments to --script,
+ those arguments seem to get lost when the Nmap command is printed in
+ Nmap's output files. For example, I run the command:
+ nmap -oN - --script=discovery,intrusive scanme.nmap.org
+ The output includes:
+ # Nmap 4.85BETA4 scan initiated Thu Mar 26 15:40:05 2009 as: ./nmap
+ -oN - --script=discovery scanme.nmap.org
+ Note the missing ",intrusive" in the script argument. [David]
+
+o Merge patrick/nse-lua-merge for easier-to-maintain and simpler
+ codebase once David and Patrick are happy with it. [David]
+
+o SVN check out /nmap as an external in a directory named svn or src
+ or nmapsvn or something under nmap.org web tree. Then redirect the
+ individual nmap.org/data/ files, where needed, to the nmapsvn
+ instead. and update nmap-dev Makefile not to copy them to the
+ /data/ dir anymore. Then update the nsedoc system to generate proper
+ links to the new script/nselib locations. [Fyodor]
+
+o Improvements to presentation of version detection
+ information. [Brandon]
+ o Allow longer strings. Right now it can be 128 chars for the
+ fullversion info, I think. But that isn't enough for this useful
+ information-packed string: "Apache httpd 2.0.52 ((Red Hat)
+ mod_perl/1.99_16 Perl/v5.8.5 DAV/2 mod_jk/1.2.19 PHP/4.3.9
+ mod_python/3.1.3 Python/2.3.4 mod_ssl/2.0.52 OpenSSL/0.9.7a)".
+ After discussion w/Brandon, we're going to allow 160 chars total.
+ o Instead of omitting all information when version info string too
+ long, we're going to truncate and allow 157 characters, plus
+ ellipses (...)
+ o Brandon says: "my final gripe is that the full version string is
+ constructed as <product><space><version><space>(<extrainfo>).
+ but, even if product or version are blank, the spaces are still
+ there"
+
+o I need an output-autoflush option of some sort. This could be
+ useful to ensure I get all the --packet_trace and debug data before
+ Nmap crashes. Actually, I'm not sure that is so critical.
+ o Killing it for now, not sure that it even is needed.
+
+o Fix the directory function(s) in nse_fs.cc to be usable by scripts and
+ improve flexibility. [this entry added by Patrick]
+
+o [Ncat] The sys_wrap.c/.h code contains a whole bunch of capitalized
+ versions of system calls (Fork(), Socket(), Sscanf(), etc.) which
+ are mostly the same as the standard version except that they cause
+ ncat to quit if they are triggered. They also may be used partially
+ for portability. The main issues are:
+ 1) Because the function quits in the case of errors, it doesn't
+ always have the context to print a useful error message (and
+ even when it does, it often doesn't -- for example Fopen could
+ print the filename, but doesn't.) Also, sometimes these
+ functions are called when quitting really isn't the desired
+ outcome of an error.
+ 2) Some could be replaced by code in nbase, for example, Malloc
+ basically does the same thing as our safe_malloc already used
+ throughout Nmap.
+ So we should probably consider simplifying/removing this code to the
+ extent possible. But we need to remember to add error detection to
+ the callers where necessary rather than blindly switching from
+ (e.g.) Connect() to connect(). [Kris or David]
+
+o With --version-trace (may be a problem with other uses of nsock
+ tracing too), I often get dozens of "wait_for_events" reports in a
+ row in a very short period, flooding the logs. For example, with
+ the command "nmap -sV --version-trace www.google.com", I get:
+ NSOCK (22.3570s) Callback: WRITE SUCCESS for EID 283 [74.125.19.147:443]
+ NSOCK (22.3570s) msevent_delete (IOD #4) (EID #283)
+ NSOCK (22.3570s) wait_for_events
+ NSOCK (22.3570s) wait_for_events
+ NSOCK (22.3570s) wait_for_events
+ NSOCK (22.3570s) wait_for_events
+ NSOCK (22.3570s) wait_for_events
+ NSOCK (22.3570s) wait_for_events
+ NSOCK (22.3570s) wait_for_events
+ NSOCK (22.3570s) wait_for_events
+ NSOCK (22.3570s) wait_for_events
+ [Goes on for pages]
+
+o NSE memory issues (and gh_list assert failure) [David]
+ o See this thread: http://seclists.org/nmap-dev/2009/q1/0532.html
+ o We're taking this out for now since the new nse-lua-merge
+ tenatively looks like it fixes this.
+
+o [Ncat] Why does Ncat require enclosure in a while loop to answer
+ repeated UDP queries, but not TCP? For example, see the "Emulating
+ Diagnostic Services" section of the Ncat user's guide.
+ o Note: http://seclists.org/nmap-dev/2009/q1/0133.html
+
+o Determine what we should do about the IE.DLI OS detection test [David]
+ o All of the 1656 results for this test in nmap-os-db are DLI=S.
+ o Is the test not working right (producing the proper results
+ against targets), or is it just a generally useless test for
+ which virtually all targets respond the same way?
+ o Are there other "useless" tests in nmap-os-db? It is worth
+ checking, IMHO.
+ o We're going to get rid of IE.DLI, IE.SI, U1.RUL, and maybe TOS and
+ TOSI tests.
+
+o When you do ncat -h, Ncat should probably show the Nmap version
+ number rather than (currently) 0.2. Also ncat in -v mode should
+ show that same header. [David]
+
+o Ncat verbose mode (-v) should probably only give important messages,
+ such as perhaps a message once you connect successfully to a port,
+ or a message if the connection attempt times out. An Ncat version
+ banner (with URL) like Nmap has might be warranted (in verbose
+ mode). Currently, Ncat floods you with (mostly) useless debugging
+ information like this with a single -v (this output, on the other
+ hand, might be useful for a debugging option): [David]
+ # ncat -C -v scanme.nmap.org 80
+ NSOCK (0.0000s) TCP connection requested to 64.13.134.52:80 (IOD #1) EID 8
+ NSOCK (0.0200s) Callback: CONNECT SUCCESS for EID 8 [64.13.134.52:80]
+ NSOCK (0.0200s) Read request from IOD #1 [64.13.134.52:80] (timeout: -1ms) EID 18
+ NSOCK (0.0200s) Read request for 0 bytes from IOD #2 (peer unspecified) EID 26
+ GET / HTTP/1.0
+ NSOCK (4.4280s) Callback READ SUCCESS for EID 26 (peer unspecified) (15 bytes)
+ NSOCK (4.4280s) Write request for 16 bytes to IOD #1 EID 35 [64.13.134.52:80]
+ NSOCK (4.4280s) Callback: WRITE SUCCESS for EID 35 [64.13.134.52:80]
+ NSOCK (4.4280s) Read request for 0 bytes from IOD #2 (peer unspecified) EID 42
+ For comparison, here is what Eric Jackson's nc (The nc available in
+ Fedora 10's package repository) shows in verbose mode for the same
+ connection:
+ # nc -v scanme.nmap.org 80
+ Connection to scanme.nmap.org 80 port [tcp/http] succeeded!
+ GET / HTTP/1.0 [David]
+
+o Final polishing of our GSoC pages. [Fyodor]
+
+o Advertise widely for Nmap GSoC applicants [Fyodor]
+
+o [Ncat] We should (maybe) consider a way for people to choose
+ usernames in --chat.
+ o Removing this for now. We can add it back if we decide we really
+ want this.
+
+o Deal with new Python 2.6 Zenmap build warnings:
+ C:\Python26\lib\site-packages\py2exe\build_exe.py:16: DeprecationWarning: the sets module is deprecated
+ import sets
+ http://sourceforge.net/tracker/index.php?func=detail&aid=2314799&group_id=15583&atid=115583
+ [Bug in py2exe, will probably be fixed with a new version of py2exe
+ once it is released and we upgrade. This isn't causing us any major
+ problem anyway.]
+
+o When I scan large groups of hosts with OS detection enabled, I get
+ groups of warnings like:
+ Insufficient responses for TCP sequencing (0), OS detection may be less accurate
+ Insufficient responses for TCP sequencing (0), OS detection may be less accurate
+ Insufficient responses for TCP sequencing (0), OS detection may be less accurate
+ Insufficient responses for TCP sequencing (0), OS detection may be less accurate
+ Insufficient responses for TCP sequencing (0), OS detection may be less accurate
+ Note how it doesn't even tell the relevant IP address, and it isn't
+ included in an individual host section. We should probably either
+ include it in the section for an individual host, like we do with
+ "OSScan results may be unreliable because we could not find at least
+ 1 open and 1 closed port", or (not quite as
+ good) include the relevant IP address in the error message. And we
+ may or may not want to require verbose mode.
+
+o Ncat chat should bomine the "already connected" user list into one
+ line, like:
+ <announce> already connected: 69.232.238.42 is connected as <user5>, 206.81.65.43 as <user4>, 69.232.238.42 as <user6>
+
+o [Ndiff] Maybe Ndiff should display changes to version detection and
+ OS detection information? [David]
+ o Version detection done, now just needs OS detection.
+
+o When I start ncat chat with this tcsh command:
+ ncat -l --chat scanme.nmap.org < /dev/null >& /dev/null &
+ The first client to connect to the chat becomes user0 and doesn't
+ work quite right. Messages user0 type get transmitted to other
+ clients, but user0 does not see their messages. Nore does user0 get
+ the normal connection announcement upon connecting. If I quit
+ user0, the next client to connect becomes user0 again and has the
+ same problem. If I start ncat on the server with "ncat -l --chat
+ scanme.nmap.org" (no redirection), other clients can connect with no problems.
+
+o Ncat --chat should probably announce to everyone (including the new
+ person) when someone connects. This tells the new person their
+ username, and lets everyone else know about the new connection. [David]
+ o We should also tell the new person (and possibly everyone on the
+ channel) the list of existing participants.
+
+o SoC ideas page [Fyodor]
+
+o Nmap 4.85BETA4 release [Fyodor]
+
+o [Ncat] Wouldn't it be nice if we could support --exec (and maybe
+ some sort of partial-emulated --sh-exec) on Windows? [David]
+ o Almost working! We found some problems with "ncat.exe -v -l
+ --sh-exec "ncat -v scanme.nmap.org"
+
+o [Ncat] Can we use it as an IPv4 <-> IPv6 gateway? If so (or if we
+ can add it), it should be added to the ncat guide feature list.
+ o Yes, David tried it with --sh-exec and it worked.
+
+o [Ncat] We should probably make it work without OpenSSL. When I try
+ ./configure --without-openssl on latest svn Nmap, Ncat build fails
+ with:
+ gcc -MM -I../libpcap -DHAVE_CONFIG_H -D_FORTIFY_SOURCE=2 -I. -I.. -I../nsock/include/ -I../nbase ncat_main.c ncat_connect.c ncat_core.c ncat_listen.c ncat_proxy.c ncat_broker.c ncat_hostmatch.c ncat_ssl.c util.c sys_wrap.c > makefile.dep
+ make[2]: Leaving directory `/mondo/fyodor/nmap/ncat'
+ make[2]: Entering directory `/mondo/fyodor/nmap/ncat'
+ gcc -I../libpcap -DHAVE_CONFIG_H -D_FORTIFY_SOURCE=2 -I. -I.. -I../nsock/include/ -I../nbase -c ncat_main.c -o ncat_main.o
+ ncat_main.c: In function ‘main’:
+ ncat_main.c:536: error: ‘struct options’ has no member named ‘ssl’
+ ncat_main.c: In function ‘ncat_listen_mode’:
+ ncat_main.c:646: error: ‘struct options’ has no member named ‘ssl’
+ ncat_main.c:646: error: ‘struct options’ has no member named ‘sslcert’
+ ncat_main.c:646: error: ‘struct options’ has no member named ‘sslkey’
+ make[2]: *** [ncat_main.o] Error 1
+ make[2]: Leaving directory `/mondo/fyodor/nmap/ncat'
+ make[1]: *** [build-ncat] Error 2
+ make[1]: Leaving directory `/mondo/fyodor/nmap'
+ make: *** [static] Error 2
+
+o [Ncat] Defensive coding review of Ncat --chat (talk)
+
+o [Ncat] As SSL server it should not crash when someone connects in
+ w/o SSL and does ^C. When David tried it during our chat, the ncat
+ servr "ncat --broker --ssl-key test-key.pem --ssl-cert test-cert.pem
+ --ssl --chat -l" crashed with: SSL_accept():
+ error:00000000:lib(0):func(0):reason(0). Also, when a Windows SSL
+ clients joined and then left, the server died with "Broken pipe
+
+o [Ncat] --chat should probably only allow reasonable chars, to avoid
+ cntrl-chars, etc.
+
+o Nmap should treat ports named "unknown" in nmap-services the same
+ way (from a naming perspective) as it treats ports which are not
+ listed at all. See http://seclists.org/nmap-dev/2009/q1/0589.html.
+
+o Ncat user guide "Emulating Diagnostic Services" page has a very long
+ UDP chargen server line which causes wrapping problems in web browsers
+ (e.g. it widens the page substantially). It should probably be
+ split into multiple lines. [David]
+
+o Ncat user guide proxying section says "The only exception is when
+ listing a proxy host by IPv6 address; then the port is required."
+ Why would we require a port number for IPv6 rather than just use the
+ same defaults as we do for IPv4?
+ [David explained that this is because to do otherwise would be
+ ambiguous because IPv6 uses : for separaters, so we wouldn't know
+ how to handle things like FF::10:80]
+
+o [Ncat] Perhaps we should make --ssl work in --chat. If nothing
+ else, it might be useful if you want to reduce the number of people
+ connecting with telnet, etc. rather than ncat.
+
+o [Ncat] --talk should probably be changed (in the code and
+ documentation) to --chat, as Ncat chat has a
+ much nicer ring to it, IMHO. --talk should remain as an alias to
+ --chat, but we don't need to document it. [David]
+
+o Ncat Windows issue where you make a connection and then take several
+ seconds to type in a line to the server, Ncat wrongly times out when
+ trying to write your line to the remote server. [David]
+
+o Ncat write timeout problems cause client to quit due to write
+ timeout sometimes. [David]
+ Examples:
+ o yes | ncat localhost
+ o when we paste a few lines into the terminal window in an Ncat chat
+
+o Defensive coding review of ncat_proxy.* [David]
+
+o Process the latest version detection submissions. We now have more
+ than 1,700 of them queued up. [Doug]
+
+o Write Ncat users' guide, demonstrating all the neat stuff you can do
+ with it. This should probably be in DocBook XML so it can be an NNS
+ chapter. You might want to query nmap-dev for list of neat things
+ people do with ncat (or look around for what people do with nc).
+ Testing it out for examples might expose areas for improvement as
+ well. [David]
+
+o Look at Dario Ciccarone's email from 5/1/07 about IPID sequence
+ issues, and consider adding IPID sequence test for closed-port-tcp as
+ they apparently can be different. [David]
+ o Also fix bug which causes SEQ to not be printed if the TCP open
+ port tests fail to produce results, even though the II and
+ (upcoming) CI tests may have useful results. [David]
+
+o NSE should offer some way to sleep/yield for a given amount of
+ time. This would allow other scripts to run while a script has
+ nothing to do. Possible uses:
+ o Many services have rate limits (or you might just want to use them
+ for politeness). For example, a web site spidering application
+ might want to limit HTTP requests to some number per second to avoid
+ pissing off the target webmaster more than is necessary (or prevent
+ getting auto-blocked). Similarly, whois servers often will block
+ IPs which query them too often in a short period. Or maybe you
+ don't want to exceed the threshold limits of an IDS.
+ o Example current scripts which might benefit: sql-injection, whois
+ (possibly), pop3-brute, etc.
+ o If we don't currently have a way for a cpu-bound NSE script to
+ yield, then perhaps this could help us implement such a mechanism.
+ But maybe coroutine.yield already does the trick.
+ o The mechanism needs to be documented, and ideally should be
+ implemented in at least one of the scripts shipped with Nmap.
+
+o Consider adding a way for requesting timing status updates at a
+ given interval (such as every 5 seconds) to XML and/or normal
+ output. This would be useful for people who run Nmap from scripts
+ or other higher level applications. [David]
+
+o Ncat --allow/--deny bug: "--allow and --deny only support host
+ specification by IP address, and give no warning when you use
+ another form such as a host name." Should probably use same syntax
+ as --exclude. We also want to at least do verification at the
+ beginning to make sure all the entries are legitimately formed. We
+ probably want to do things like DNS resolution at the beginning
+ too. Otherwise we might have a DNS failure when we actually get a
+ connection and perhaps have to reject the connection wrongly, or
+ risk a false negative. [David]
+
+o Fix this overflow:
+ Stats: 93:57:40 elapsed; 254868 hosts completed (2048 up), 2048 undergoing UDP Scan
+ UDP Scan Timing: About 11.34% done; ETC: 03:21 (-688:-41:-48 remaining)
+ [Done by David and Henri Doreau]
+
+o Ncat -- perhaps connection brokering should support UDP as well as
+ (its existing support for) TCP? Actually this does raise issues
+ such as deciding what list of UDP systems to forward a packet too.
+ Its obviously not like TCP where you have a list of open
+ connections. Ncat could build such a list, but, for example, would
+ never know when to remove the host. For now, David is just going to
+ adjust the error message to encourage people to email nmap-dev
+ describing their usage scenario if they want this feature.
+
+o Ncat documentation should note that no SSL certificate verification
+ is done (maybe we should offer an option to do so, if OpenSSL makes
+ that easy).
+ o Done in the new Ncat user's guide
+
+o Fix dns-zone-transfer infinite recursion bug described at
+ http://seclists.org/nmap-dev/2009/q1/0317.html. It sounds like the
+ best approach is to use our dns.lua library rather than having
+ dns-zone-transfer do its own DNS packet parsing.
+
+o Fix XML escaping issue so that improper chars from NSE scripts or
+ elsewhere can't cause corrupt XML files. See
+ http://seclists.org/nmap-dev/2009/q1/0316.html for an example. [David]
+
+o Look into whether we should increase the frequency of port scan
+ pings. See http://seclists.org/nmap-dev/2008/q1/0096.html . Note
+ that Fyodor already increased them a bit in 2008. Might not need
+ more. [David did extensive testing of this one already]
+
+o Find way to document NSE library script arguments and perhaps have
+ them bubble up to scripts themselves. For example, I had to read
+ the SNMP library source code to determine the script argument to
+ specify the SNMP community name for snmp-sysdescr
+ (https://nmap.org/nsedoc/scripts/snmp-sysdescr.html). Maybe we could
+ just standardize on something like we do with SMB library and the
+ scripts which call it (https://nmap.org/nsedoc/modules/smb.html,
+ https://nmap.org/nsedoc/scripts/smb-check-vulns.html). [David]
+
+o If it wouldn't bloat things too much, it would be nice to include
+ ndiff in the Nmap win32 zip distribution files.
+
+o Reported NSE crash:
+ "Assertion failed - file ..\nse_main.cc line 314
+ lua_gettop(L_script_scan) == 0"
+ o He says: "After looking at this closer, it appears the assertion
+ occurs if I include the IP where the scan is run from. For us, I'm
+ running this on IP 57, which is a VMware Windows Server image. If
+ I eliminate that IP from the range it successfully completed the
+ scan for all other devices."
+ o Seems to be fixed. He can no longer reproduce the problem with
+ 4.85BETA3.
+
+o Deal with GTK DLL problem with Nmap 4.85BETA1: [Fyodor]
+ o David's installer seems to work--he's using a different GTK
+ distribution. I'll try that. Works! Done!
+ o Details on problem: http://seclists.org/nmap-dev/2009/q1/0207.html
+ o Quick workaround done for 4.85BETA2, but better solution needed.
+
+o "SCRIPT ENGINE (250.600s): ./scripts/rpcinfo.nse against
+ a.b.c.d:<port> ended with error: ./nselib/datafiles.lua:114: attempt
+ to index global 'arg' (a nil value)"
+ -- http://seclists.org/nmap-dev/2009/q1/0227.html [Patrick]
+
+o Consider making the TODO list public
+ o Done: http://seclists.org/nmap-dev/2009/q1/0175.html
+ o Probably remove all of the "done" items since that is easier than
+ reviewing them.
+ o Might as well add to insecure.org/nmap/data/
+ o Maybe a bug tracker is a better approach.
+
+o [NPING] Fix compilation on Solaris. See
+ http://seclists.org/nmap-dev/2010/q1/870.
+
diff --git a/todo/gorjan.txt b/todo/gorjan.txt
new file mode 100644
index 0000000..3eada09
--- /dev/null
+++ b/todo/gorjan.txt
@@ -0,0 +1,66 @@
+=====
+GSoC 2011 participation: Discovery and miscelaneous script specialist
+=====
+
+Work in progress:
+
+* bgpmon-info analyze
+
+* bittorrent-dht-nodes
+
+* lldp - write script proposal
+http://en.wikipedia.org/wiki/Link_Layer_Discovery_Protocol
+
+* disjunctive-traceroute analyze feasibility
+http://ccr.sigcomm.org/online/?q=node/398
+
+=====
+
+ToDo:
+
+* snmp-brute port to brute framework
+There are a couple of default passwords that snmp-brute uses atm which should be
+considered even when it's the brute.lua is used
+
+=====
+
+Maybe (the ones with ** aren't on the Script_Ideas Page yet)
+
+* Bonjour / mdns / llmnr etc.
+(DNS protocols support) + backscatter into dns scripts where applicable?
+
+* targets-asn
+John Bond is working on this. It's called asn-to-prefixes. Perhaps I could
+review it, asist so it makes its way to the library faster? On the other hand
+there already are a couple of people assisting.
+
+* targets-dhcp
+dhcp-discover as a prerule, so it doesn't run by default. But it doesn't run by
+default. It's discovery, intrusive, but not default. Maybe just add the prerule
+there, and some way of forcibly initiating the prerule (like an argument).
+
+* hnap-info
+* hnap-auth-bypass
+A nice hnap library would be fitting, that will make these scripts a breeze.
+I'd need testing equipment, or some :S implementation.
+
+* vuze-dht-version
+* Nbstat.nse -> change to using a broadcast prerule
+* SSL renegotiation
+* soap.lua
+* xmlrpc.lua
+
+=====
+
+Completed:
+
+* broadcast-ping
+* nmap lib: get_ttl() and get_payload_info()
+* ip-geolocation scripts
+* snmp-interfaces patch related to mac-geolocation
+* mac-geolocation
+* stdnse.lua: in_port_range()
+* backorifice-brute
+* backorifice-info
+
+===== \ No newline at end of file
diff --git a/todo/henri.txt b/todo/henri.txt
new file mode 100644
index 0000000..cca8814
--- /dev/null
+++ b/todo/henri.txt
@@ -0,0 +1,41 @@
+o Proper SSL support in proxy mode.
+ - A naive implementation relying on the current code would probably look
+ horrible (at least my own attempts did). I believe that nsock should
+ internally be able to SSLify a plain TCP connection. It doesn't have to be
+ exported but it should be implemented just like the other operations. Then
+ it would be trivial (and clean) for the library to SSLify the channel
+ established by the proxy hooks.
+ - When redesigning nsock SSL code, keep in mind the ability to establish a SSL
+ session and still expose the raw TCP. That can be convenient when auditing
+ the SSL/TLS layer.
+o Don't drop pending writes when deleting the corresponding IOD. For nsock to
+ behave a bit like standard BSD sockets we should flush writes on close. (OTOH
+ anything which isn't ack'ed has no meaning, caller can still cancel it
+ typically...)
+o Give IODs their own methods to streamline the code and get rid of all
+ the special cases in nsock_core.c. This would also make it easier to
+ hook operations (typically: override the default iod_connect() method
+ to establish a proxy chain).
+o Fix the read API (!)
+o Profile the pcap code. It needs cleanup (for sure) and optimizations (maybe).
+o Proxy authentication
+o Handle socks4a
+ - This requires to figure out how to trigger proxy code without
+ resolving target hostname first. The problem is that the proxy code
+ is supposed to be a transparent hook of connect()... Extending the
+ exported API will probably be needed :(
+ - Async hostname resolution available from within nsock would let us
+ try clever tricks... I'm not sure whether nsock should provide it
+ or if it should simply provide an API to plug an external system.
+o Socks5 support
+o Some code is copied from ncat. I should move it to nbase.
+o Replace event lists by more efficient data structures. Consider using
+ a radix tree to map event IDs to pointers. Another solution would
+ be to put them all into a single RB-tree (TODO: validate BSD_HACK_MODE
+ & stuff). Encoding the event type in the ID's MSB would let us do inorder
+ traversal with connect events first, then read, then write...
+ {NOTE: It'd be cool for the beauty of it, but my tests reveal that as of Oct.
+ 2013 there's no big bottleneck there.}
+o Rework the filespace code to avoid unneeded data copy. Scatter/gather
+ I/O might be useful there. Same task can also be expressed as: "profile and
+ optimize the usual nmap nsock I/O patterns."
diff --git a/todo/nmap.txt b/todo/nmap.txt
new file mode 100644
index 0000000..76fe1ff
--- /dev/null
+++ b/todo/nmap.txt
@@ -0,0 +1,638 @@
+TODO $Id: TODO 11866 2009-01-24 23:10:05Z fyodor $ -*-text-*-
+
+o Work on Nmap on Mobile devices, particularly Android. Would be
+ great to get it in Google Play store, for example. An official
+ version with a workable GUI. For now, people have to do manual work
+ and it isn't as well tested either:
+ https://secwiki.org/w/Nmap/Android . If this is successful, we could
+ consider iOS.
+
+o Nmap performance work. Particularly with --min-rate.
+
+o Consider re-architecting Nmap to have more of a scanning pipeline
+approach rather than fixed sets of hosts which start and finish one
+phase and then move into the next in parallel. This could potentially
+allow us to add hosts one by one to a phase as other hosts finish that
+phase and, ideally, the phases could run in parallel too.
+
+o Nmap Network Scanning, 2nd Edition work [placeholder]
+
+o Organize nselib into a hierarchy. A library "dirname/filename.lua" can be
+ required as "dirname.filename". We would need to ensure the installers
+ (Makefile, OS X, Windows, RPM) can handle this. See
+ http://seclists.org/nmap-dev/2014/q3/364
+
+o We should work to reduce Zenmap's memory consumption. We used to
+ commonly get error reports from people who load so many systems that
+ Zenmap gives an out of memory error and crashes. For example, see
+ this thread: http://seclists.org/nmap-dev/2014/q2/46
+ After committing patch at http://seclists.org/nmap-dev/2014/q2/429,
+ we no longer get the error report but the problem still exists.
+ The problem seems to lie in a very large Nmap Output being stored
+ in memory and a possible fix seems to be to use a file based paging
+ system.
+
+o Consider making a version of Nmap for Apple's official Mac App
+ Store. A particular concern with the downloadable Mac version of
+ Nmap is that Apple's new "Mountain Lion" release may require users
+ to jump through hoops to install unsigned non-app-store content per
+ their "Gatekeeper" "feature". Though maybe signing the app will be
+ enough. There may also be an issue with the "Sandboxing"
+ requirement for App Store apps starting June 2012. Will Nmap be
+ able to request all the permissions it needs? Ignoring the
+ technical challenges for the moment, what will users prefer?
+
+o Do a roll up on (state, TTL) pair instead of just state so that TTL
+ info is not lost when doing roll up on port states.
+ See thread at http://seclists.org/nmap-dev/2014/q3/93
+
+o Consider looking into differring TTL values during OS detection
+ phase and choose a port that is (hopefully) not firewalled to get
+ a better chance at correct result. See thread at
+ http://seclists.org/nmap-dev/2014/q3/33
+
+o [Zenmap] Look into and refactor code which uses the (very slow) += operation
+ on strings. http://seclists.org/nmap-dev/2014/q2/432 helped improve speeds
+ for opening files (from hours to seconds) and it seems like more speedups
+ can be done in other places.
+
+o Look into moving our Mac building/testing system into a virtual
+ machine or leased server sort of environment so that multiple Nmap
+ developers can access it and nobody has to keep a stack of Mac Minis
+ in their closet.
+
+o INFRASTRUCTURE: Upgrade seclists to use Mailman 3, which apparently
+ has many improvements.
+
+o We should fix nsedoc generation so it doesn't fail when blocks like
+ @usage, @output, etc. are followed by a local declaration. See
+ http://seclists.org/nmap-dev/2014/q2/331. If for some reason this
+ just can't be fixed, we will have to document the heck out of it, I
+ suppose.
+
+o When scanning your own IP from Windows, Nmap currently recognizes
+ the problem (can't do a raw scan like that on Windows) and skips the
+ SYN scan, leading to Nmap printing a bunch of ports in "unknown"
+ state at the end. Nmap should probably act like unprivileged mode
+ in this case (e.g. do a connect scan, etc.). See
+ http://seclists.org/nmap-dev/2013/q3/519
+
+o Investigate Checkmarx static analysis report of Nmap source tree
+ that someone sent us on Feb 12. It looks like mostly false positives,
+ but we should go through to check for any real bugs or even possible
+ security issues. Fyodor has the report.
+
+o INFRASTRUCTURE: Consider updating our svn-mailer.py (and conf file)
+ to the latest official version. First check whether there is a
+ later official version and whether it has material changes. We're
+ currently using one from
+ subversion-1.4.2/tools/hook-scripts/mailer/mailer.py.
+
+o Consider a two-stage model for IPv6 subnet/pattern support
+ o Right now you can try to scan a /64, for example, and Nmap will try
+ to iterate through them all (and of course never complete). So
+ perhaps Nmap should first look at a specification and decide if it
+ should use other techniques like multicast discovery instead.
+
+o Move advanced IPv6 host discovery features from NSE into core Nmap.
+ We'll probably add the functionality of
+ targets-ipv6-multicast-invalid-dst, targets-ipv6-multicast-echo, and
+ maybe targets-ipv6-multicast-slaac.
+ - The idea is that Nmap does them automatically if it gets a large
+ target specification and sees that it is local so can be multicast
+ pinged.
+
+o We should figure out why (at least with Nping) raw ethernet frame
+ sends seem to be taking significantly longer than raw socket sends
+ (e.g. using --send-ip or the OS-provided ping utility). This has
+ been reproduced on Linux and Windows. Here's a thread:
+ http://seclists.org/nmap-dev/2012/q4/424
+ o Note that David and I tried to reproduce this on his machine and
+ on 'web' and 'research' machines and could not reproduce. Still
+ happens with Fyodor's machine connected with WiFi. Fyodor should
+ test on the same machine using wired and see if that changes anything.
+
+o Implement some improvements to dns-ip6-arpa.nse, as describe at
+ http://seclists.org/nmap-dev/2012/q2/45.
+ - Also consider a move to "fire and forget" logic. Just blast out
+ the queries that we know we have to make, and then read any replies
+ that may happen to come back. (but still try not to introduce
+ inaccuracy (missed hosts) by flooding the network.
+
+o Treat the input to the escape function in xml.cc as UTF-8, not just
+ ASCII. Good UTF-8 should survive into the output; i.e., "\xe2\x98\xbb"
+ should become "\xe2\x98\xbb" in the output, not "&#xe2;&#x98;&#xbb;".
+ If the input happens not to be UTF-8, (like the file name in
+ http://seclists.org/nmap-dev/2013/q1/180), I suppose we can
+ individually encode each byte of each invalid sequence: "\xba\xda\xbf"
+ becomes "&#xba;&#xda;&#xbf;". Can probably do this with simple
+ byte->rune and rune->byte functions as in
+ http://plan9.bell-labs.com/sys/doc/utf.html.
+
+o We should probably redo the Nmap header (e.g. on https://nmap.org) to
+ make it more attractive. Or, at a minimum we should update the
+ screenshots and think about which links we really need (some of those
+ pages aren't really updated any more).
+
+o Test a hierarchical classifier for IPv6 OS detection. Our classifier
+ currently treats, for example, some localhost Linux fingerprints as
+ separate classes from remote Linux fingerprints, simply because we
+ lose precision if we lump them together (for example TCP window size
+ differs across certain Linux versions when measured remotely, but
+ not on localhost). This leads to the linear classifier having to use
+ narrow margins between fingerprints that are really very similar. I
+ want to try a tree of classification where each non-leaf node is a
+ separately trained classifier and each leaf node is a final
+ classification. The first layer of the hierarchy would be something
+ like
+ (linux windows solaris aix ... other)
+ where "linux" would contain *all* the Linux fingerprints in a single
+ class. Lower levels would be like
+ (linux-2.4 linux-2.6)
+ (windows-xp windows-vista windows-7)
+ Lower levels will include only those fingerprints in their parent
+ class, so we don't even think about Windows when classifying
+ Linux. Probably three or four levels will be sufficient. There may
+ be a principled or automatic way to build this hierarchy, but I
+ suspect playing it by ear will be sufficient. Talk to David for more
+ of his thinking on this topic.
+
+o Maybe we should rename dns-brute to dns-brute-enum since it is so different
+ from our traditional brute force authentication cracking -brute scripts?
+
+o NSE WORK (note that this is mostly infrastructure because script
+ ideas are generally put on the script ideas page instead:
+ https://secwiki.org/w/Nmap_Script_Ideas)
+ o Review NSE-based port scanning and RST idle scan.
+ http://seclists.org/nmap-dev/2011/q2/307. [Henri and Hani?]
+
+o Maybe we should add an analysis or reporting or intelligence (or
+ different name) for our NSE scripts which don't send any packets, but
+ simply analyze Nmap's existing data and report when useful.
+
+o Install some sort of svnview webapp for svn.nmap.org which is
+ wrapped in Insecure chrome, allows people to click link for direct
+ file download, probably shows revision history and allows users to
+ see older versions, etc.
+
+o Process Nmap survey and send out results [Fyodor]
+
+o Nping (we think) will stop after 2^32 rounds even when "-c 0" is
+ given. We should probably make this a 64-bit integrer so that "-c
+ 0" will go essentially forever and so that users can give values
+ higher than 4 billion.
+
+o Nscan work [placeholder]
+ - Hosted Nmap system
+
+o Add CPE entries to OS fingerpting DB entries which still lack them.
+ This is a gradual process since almost all of the missing ones
+ aren't in the official CPE dictionary either and it can take a lot
+ of research to decide on an appropriate entry. Milestones so far:
+ - 3/21/12: We have entries for 2,601 of 3,572 fingerprints (971
+ missing; 73% coverage)
+ - 11/5/12: We have entries for 3,285 of 3,907 fingerpritns (622
+ missing; 84% coverage)
+ - 11/12/12: We have entries for 3,558 of 3,946 fingerprints (388
+ missing; 90% coverage).
+
+o [Zenmap] should actually parse and use script results. See
+ http://seclists.org/nmap-dev/2010/q1/1108
+ - We have an initial prototype, but probably need to redo because it
+ doesn't present the results in the way we'd like yet due to
+ problems implementing such a presentation with GTK, etc.
+
+o Make Zenmap settings get upgraded when the Zenmap executable is
+ upgraded. The per-user configuration files such as scan_profile.usp
+ and zenmap.conf are never overwritten once installed by Zenmap, so
+ changes and fixes to those files don't reach anyone who has
+ installed Zenmap already. This is most noticeable with changes to
+ profiles and highlight definitions are notably affected. This fix
+ may involve hard-coding settings that are not normally configured by
+ users (like highlighting) or updating the per-user files at startup
+ (only those parts that haven't been changed by the user).
+
+o We should offer partial results when a host timeouts. I (Fyodor)
+ have been against this in the past, but maybe the value is
+ sufficient to be worth the maintenance headaches. Many users have
+ asked for this. If we do implement this, we may want to only print
+ results for the COMPLETED phases (e.g. host discovery, port
+ scanning, version detection, traceroute, NSE, etc.) Trying to print
+ partial results of a port scan or NSE or the like might be a pain.
+ And if we print some results for a host which timeouts, we should
+ give a very clear warning that the results for that host are
+ incomplete. As an example, here is someone who hacked Nmap source
+ code to achieve this: http://seclists.org/pen-test/2010/Mar/108.
+ o Another benefit would be that it would allow us to clean
+ up/regularize the host output code. Right now there are I think
+ three places where a host's final output can be printed. If,
+ instead, that code just looked at what information was available and
+ printed that out only, we could potentially isolate it in just one
+ place.
+ o This also might let us provide a feature for skipping the rest of
+ an Nmap phase which is going too slowly (I think that has its own
+ Nmap TODO item).
+
+o [Nsock] Some SSL connections that used to work now fail; find out
+ why. http://seclists.org/nmap-dev/2010/q4/788. Narrowed down to
+ r19801 in http://seclists.org/nmap-dev/2011/q1/12.
+
+o [NSE] Consider a system where scripts can tell if any other scripts
+ depend on them. They could then use that to determine whether they
+ should bother storing information in the registry. For example,
+ snmp-interfaces could store the discovered table if another script
+ (such as a mac address geolocator script) depends on it.
+
+o [NSE] Consider whether we need script.db for performance reasons at
+ all or should just read through all the scripts and parse on the fly.
+ See: [http://seclists.org/nmap-dev/2009/q2/0221.html]
+
+o A couple minor nsedoc issues (see
+ http://seclists.org/nmap-dev/2011/q1/1095):
+ o After the ssh-hostkey portrule was added, nsedoc seems to be
+ generating a blank "Script types" filed for the script:
+ http://localhost:8082/nsedoc/scripts/ssh-hostkey.html
+ o This is happening because "portrule" and "hostrule" appear later in
+ the script, and NSEDoc thinks it is their definition, and there is
+ no NSEDoc there.
+ local ActionsTable = {
+ -- portrule: retrieve ssh hostkey
+ portrule = portaction,
+ -- postrule: look for duplicate hosts (same hostkey)
+ postrule = postaction
+ }
+ o ssh-hostkey and rmi-dumpregistry each have two @output sections,
+ and NSEDoc is only showing the second one. We should probably just
+ combine them into one @output section, and maybe make nsedoc give a
+ warning in this case. Or we could make nsedoc handle multiple
+ @outputs.
+
+o Add general regression unit testing system to Nmap
+ o David has created a system for Ncat which could serve as a
+ model.
+
+o Make version detection and NSE timing system more dynamic so that
+ the concurrency can change based on network conditions/ability.
+ After all, beefy systems on fast connections should be able to handle
+ far more parallel connections than slower systems.
+ o At a minimum, this at least warrants more benchmark testing.
+
+o We should run at least one SCTP service on scanme. Daniel
+ Roethlisberger has made available dummy services which support IPv4
+ and IPv6 (see http://seclists.org/nmap-dev/2011/q2/450).
+ Alternatively, we could run some sort of "real" SCTP application(s)
+ (preferably one which is relatively simple, easy to install, secure,
+ and supports IPv6).
+
+o Create new default username list:
+ http://seclists.org/nmap-dev/2010/q1/798
+ o Could be a SoC Ncrack task, though should prove useful for Nmap
+ too
+ o We probably want to support several lists. Like an admin/default
+ list like "root", "admin", "administrator", "web", "user", "test",
+ and also a general list which we obtain from spidering from
+ emails, etc.
+
+o Improve Nsock proxies system
+ - Add SSL support
+ - Add proxy authentication
+ - Switch Ncat to using Nsock proxy system rather than it's own
+ built-in support.
+ - Move the code which is shared with ncat to nbase (URL parsing code,
+ for instance).
+ - Add socks4a/socks5 support. This requires to figure out how to
+ enter the nsock proxy code w/o having the target IP address. No huge
+ technical blocker there though, only design choices to make.
+ - Nping could potentially use it as well (could be useful for
+ measuring latency and reliability of a given proxy chain, for
+ example).
+ - Add proxy support to connect() scan. This would mean moving
+ connect scan to nsock.
+
+o [NCAT] Send one line at a time when --delay is in effect. This is
+ cumbersome to do until Nsock supports buffered reading.
+
+o [NCAT] Make the HTTP proxy support the chunked transfer encoding,
+ then change it to be HTTP/1.1 and support pipelining.
+
+o [NCAT] Drop privileges once it has started up, bound the ports it
+ needs to, etc.
+
+o [NCAT] Work as a SOCKS4a/SOCKSv5 proxy.
+
+o [NCAT] Resolve names through the proxy when possible.
+ http://seclists.org/nmap-dev/2012/q2/768
+
+o [NSE] Script writing contest (something to think about)
+
+o We should document an official way to compile/test refguide.xml so
+ people can more easily test their changes to it. This will probably
+ involve moving legal-notices.xml into /nmap/docs, among other
+ things.
+ o Note that nping has its own /nmap/nping/docs/genmanpage.sh - we
+ could look at how that could apply to Nmap.
+
+o Move Zenmap man page from nmap/docs/ to nmap/zenmap/docs to match
+ the man page location for ncat and ndiff.
+ o Don't break packaging/build system
+ o Don't break the system for posting html to web site.
+ o Consider standardizing names for nping and ncrack man pages as well.
+ [Fyodor]
+
+o [NSE] MSRPC - Improve domain support all around -- in particular,
+ let the user give the domain in the format DOMAIN\username or
+ username@DOMAIN anywhere that usernames are accepted. Suggested
+ at http://seclists.org/nmap-dev/2010/q2/389
+
+o [NSE] Combine similar MSRPC scripts, especially the "get info"
+ stuff. See this thread on combining
+ (http://seclists.org/nmap-dev/2010/q1/1023). This was suggested by
+ Ron at http://seclists.org/nmap-dev/2010/q2/389.
+
+o [Zenmap] Investigate getting new OS icon art. See
+ http://seclists.org/nmap-dev/2010/q1/1090
+
+o We should probably enhance scan stats--maybe we can add a full-scan
+ completion time estimate? Some ideas here:
+ http://seclists.org/nmap-dev/2010/q1/1007
+
+o [NSE] Do some benchmarking of our brute.nse. We should check the
+ performance with different levels of thread parallelism. Our
+ initial results show that it isn't helping much for vnc-brute or for
+ drda-brute (which is currently using the multi-thread feature
+ directly rather than through brute.nse library). We should figure
+ out why the threads aren't helping more, and whether there is
+ something we can do to fix it. It would also be interesting to
+ compare speed with Ncrack for services we have in common.
+
+o Start project to make Nmap a Featured Article on Wikipedia.
+ - See http://seclists.org/nmap-dev/2010/q1/614
+
+o Add Nmap web board/forum
+ - First step is looking at the available software for this.
+ - Nmap subreddit exists: https://www.reddit.com/r/nmap
+
+o [Zenmap] Consider a couple ideas from Norris Carden
+ (http://seclists.org/nmap-dev/2010/q2/228):
+ - remember last save and/or open location for new saves and/or opens
+ - default save location option
+
+o [Nsock] Consider adding server support to Nsock so it can accept
+ multiple connections and multiplex the SD's, like it does for
+ clients. This could potentially be used by Ncat and Nping echo
+ mode. Currently Ncat server doesn't use Nsock at all, while Nping
+ echo mode basically polls, repeating a loop of 1s in nsock_loop
+ followed by a nonblocking accept(). Then Nping gives the SD's to
+ Nsock to manage.
+
+o Consider implementing both global and per-host congestion control in
+ the IPv6 OS detection engine. Currently it handles congestion globally
+ (one CWND and SSTHRESH shared by all hosts). This works fine but it
+ may not be the most efficient approach: if the congestion is not
+ in our network segment but in a target's and we are os-scanning
+ hosts in different networks, then all hosts get "penalized" because
+ there is congestion in another network, not in theirs.
+
+o [Nsock] Consider implementing a nsock_pcap_close() function or making
+ nsp_delete() call pcap_close() when pcap IODs are used. Currently valgrind
+ warns about a socket descriptor left opened (at least in Nping).
+ ==10526== at 0x62F77A7: socket (syscall-template.S:82)
+ ==10526== by 0x4E348A5: ??? (in /usr/lib/libpcap.so.1.0.0)
+ ==10526== by 0x4E36819: pcap_activate (in /usr/lib/libpcap.so.1.0.0)
+ ==10526== by 0x4E375FC: pcap_open_live (in /usr/lib/libpcap.so.1.0.0)
+ ==10526== by 0x4311A9: nsock_pcap_open (nsock_pcap.c:64)
+ ==10526== by 0x428078: ProbeMode::start() (ProbeMode.cc:329)
+
+o Consider rethinking Nmap's -s* syntax for specifing scan types
+ o Current problems with this -s syntax:
+ o We already use like 20 of the 26 letters, so we end up with
+ things like SCTP scan using -sY
+ o Can make Nmap command lines hard to read, particularly given
+ that we often need to improvise to find a letter which isn't
+ taken.
+ o Problematic for scan types -sI and -b which require arguments
+ o Inconsistencies. For example, -sC and -sV do script scan and
+ version detection, respectively, and yet for OS detection we use
+ -O. Also, control flow (-sP, -sL) is used with -s, which further
+ overloads the options.
+ o Possible solution:
+ o We are enabling -Pn and -sn as preferred notations for -PN and
+ -sP which mean "no ping" and "no port scan". Those match the
+ already existing -n for "no DNS". The problem with -sP is that it
+ implies "ping only", when what it really should mean is "disable
+ port scan" because you may want to do NSE, OS detection,
+ traceroute, etc. still.
+ o We might want to just give them normal option strings, so you
+ could do --maimon instead of -sM, for example. For extremely
+ common options such as SYN scan, UDP scan, version detection, we
+ could perhaps find good single letter options as an alias to the
+ longer one.
+ o Another idea is to use something like --scantype syn,udp,sctp,
+ which is a lot longer for single-type scans, but shorter when
+ you're combining mulitiple ones. Doesn't allow for individual
+ scan arguments easily. I (Fyodor) think I prefer the idea above
+ of just givem them top level arguments.
+ o If we keep -s*, we could just give it one defined function, such
+ as selecting port scan type, or control flow.
+ o Obviously this will take some discussion/brainstorming on nmap-dev.
+
+o Do -p- Internet UDP scans.
+
+o Scanning through proxies
+ o Nmap should be able to scan through proxy servers, particularly now
+ that we have an NSE script for detectiong open proxies and now that
+ Ncat can act as proxy client or server.
+ o Requirements:
+ o Would be nice to be able to chain through multiple proxy servers of
+ different types.
+ o Would be nice to be able to spread the load amongst multiple
+ proxies.
+ o Should support port scanning, version detection, and NSE. In
+ other words, nsock should support proxies.
+ o Support IPv4 and v6
+ o Need to figure out how to get good performance. Pool of
+ connections to proxy or proxies for concurrency? HTTP pipelining?
+ o Support the different varieties of proxies: socks4, socks4a,
+ socks5, HTTP GET (if possible), HTTP CONNECT. Note that GET
+ proxies present some challenges since the error messages may not
+ be standard, etc.
+ o Maybe auto-detect the proxy type so that Nmap can try the most
+ efficient scanning method first?
+ o I've been asked to support basic, ntlm, and digest authentication
+ if possible.
+ o Implementation ideas:
+ o There is a patch by Zoltan Panczel (http://nmap-dev.fw.hu) and it
+ has been improved by Jacob Appelbaum in nmap-exp/ioerror/ . This
+ patch doesn't handle things like parallelization, but it may be a
+ good proof of concept.
+ o This might not be appropriate for ultra_scan ... perhaps would be
+ better to write a general scanning engine for abusing
+ applications for port scanning purposes. This could handle
+ scanning through proxies and the existing FTP bounce scan would
+ also be ported to this engine (or, frankly, we could probably get
+ away with removing FTP bounce). rembrandt at jpberlin.de tells me
+ that you can also do this with the "forwarding" commands on IMAP
+ servers. Whoever does this should probably start by reading the
+ code for the main port scanning engine (ultra_scan()) and also
+ the version detection code (service_scan()). And the version
+ detection paper at https://nmap.org/book/vscan.html. If you
+ understand all that, you may be ready for this project :). This
+ is important, because it is easy to do poorly. The tough part is
+ high performance and clean code which is general enough that all
+ these different applications can be scanned through using the
+ same basic engine. You should run your ideas by nmap-dev in as
+ much detail as possible before starting.
+ o David: I'm starting to think about building proxy support into
+ Nsock and then implementing -sT with Nsock instead of ultra_scan.
+
+o [Web] Consider adding training/introduction videos to the Nmap site
+ o Would be great to have a (5 minute or less) promotional video
+ introduction to each tool (Nmap, Zenmap, Ncat, Ndiff) on its web
+ page.
+ o They need to be good to be useful--the sort of the quality you see
+ in Laura Chappell's Wireshark videos or James Messer's Nmap videos
+ or Irongeek's videos (http://www.irongeek.com).
+ o Besides the promotional videos, users would probably enjoy more
+ in-depth video instructions (e.g. covering the Nmap Network
+ Scanning topics).
+ o Here's an example product page with lots of videos (we may not go
+ that far): http://www.splunk.com/product
+
+o The Zenmap translation system
+ (https://nmap.org/book/zenmap-lang.html) has been pretty successful
+ so far. We should consider doing the same for Nmap. After all, we
+ already have the reference guide in 16 languages at
+ https://nmap.org/docs.html. We should definitely try to use the same
+ translation methods for Zenmap as we do for Nmap. In fact, maybe we
+ can create a combined PO file Nmap, Zenmap, Ncat, and Ndiff so that
+ they can all be translated and maintained together. Something to
+ consider: calling setlocale can change the behavior of functions like
+ isalpha. Locale-dependent functions need to be checked for security
+ risks.
+
+o [NSE] Consider whether we should include some sort of NSE debugger. Or we
+ could include something simpler. For example, Nmap now provides a
+ traceback (with sufficient debugging/verbosity) when a script ends
+ in error. For some inspiration/ideas, look at Diman's NSE
+ debugger (http://seclists.org/nmap-dev/2008/q1/0228.html).
+
+o [NSE] Support routing http requests through proxies.
+
+o [NSE] Would be great if NSE scripts could be made to NOT
+ run as root if they don't have to.
+
+o [NSE] Security Review
+ o Consider what, if any, vulnerabilities or security risks NSE has
+ with respect to buffer overflows, format string bugs, any other
+ maliciously formatted responses from target systems, etc. Maybe
+ address the known risk of malicious scripts too.
+ o Consider that NSE runs scripts as root
+
+o More security auditing of Nmap code (it never hurts to do more proactive
+ security auditing).
+
+o Figure out and document (in at least the Ncat user's guide) the best
+ way to use Ncat for chaining through proxies. One option is this
+ sort of thing:
+ ncat -l localhost 1234 --sh-exec "ncat --proxy A.A.A.A B.B.B.B"
+ ncat --proxy localhost:1234 C.C.C.C
+ If you had two proxies A.A.A.A and B.B.B.B, connecting to C.C.C.C.
+ With another listener/--sh-exec pair for each additional proxy.
+ But perhaps we can make it easier by adding it to the syntax.
+
+o Look into whether we should loosen/change the global congestion
+ control system to address possible cases of one target host with many
+ dropped packets slowing down the whole group. See
+ http://seclists.org/nmap-dev/2008/q1/0096.html .
+ * Related possibility: Fix --nogcc to gracefully handle ping scans.
+ Right now it seems to go WAY TOO FAST (e.g. several thousand
+ packets per second on my DSL line).
+ * [12/22/09] David says: It still is in one case that I've
+ documented on my wiki. I had an idea to fix it, but on testing it
+ it didn't work. The idea was to treat the global congestion limit
+ differently. Instead of dropping it down to the minimum level on a
+ drop as is done currently, I thought about only dropping it by the
+ amount that the individual host limit drops. For example, if a
+ host had a drop and its limit fell from 25 to 1, then the global
+ limit would change (if it was at 100 to begin with) to 76, not all
+ the way down to 2 or whatever it is. The idea being that the
+ global limit is most important at the beginning of a scan, when
+ there's no information to set host limits, and every host wants to
+ send all its first probes at once. See
+ http://www.bamsoftware.com/wiki/Nmap/PerformanceNotesArchive2#global-cc. I
+ am convinced, though, that some sort of global control is
+ necessary. There's a reason that a web browser limits the number
+ of connections it will make, and doesn't try to download every
+ image file at once and count on the fairness of TCP to sort it
+ out.
+
+o libnmap organization for UNIX and Windows
+ o Then change Nmap and Zenmap to simply call this library
+ o It is interesting to look at: http://www.gnupg.org/gpgme.html
+
+o Deal with UDP retransmission for version detection (I think I
+ should just do a second run of all probes for UDP if it fails to
+ match anything). The advantage there is that no retransmissions are
+ neccessary if the service is found. Then again, per-probe
+ retransmission would let us redo the most likely probes (the one(s)
+ that match the port number) quickly. Lost packets should probably
+ affect ideal_parallelism.
+
+o Make RPM relocatable (requires somehow avoiding storing paths in the
+ binary)
+ - That may be easier now that David has made some big improvements
+ in detecting where the binary is cross-platform and then looking for
+ data files based on that location.
+
+o Nmaprc-related - Create a system to store Nmap defaults/preferences
+ in an nmaprc file.
+ o nmaprc should be in ~/.nmap on UNIX
+ o On Windows, we may need a registry key to find the .nmaprc
+ o Perhaps Lua could be used as the format?
+ o .nmaprc for keeping defaults, etc.
+ o Nmaprc infrastructure, hook to new timing variables
+ o Nmaprc man page
+ o Default timing mode
+ o Default NSE arguments, such as user agent
+ o Maybe Default source IP (-S) argument
+ o should be a way to specify your own .nmaprc
+ o Maybe lets you add a directory and template for saving all
+ scans.
+ o Maybe let you define "scan profiles" like is done with Zenmap.
+ There would then be a command-line option to select the profile used.
+
+o Get new Zenmap logo
+ o consider putting back on top-right of command constructor wizard
+ (there used to be umit logo there).
+ o Maybe that can be done after the release by soliciting ideas.
+
+o Create or collect some great ./configure ascii art.
+
+o Look at all the pcap functions, there are some like
+ pcap_findalldevs() which could be quite useful. There are mails to
+ the Nmap list relating to suggested improvements --
+ http://seclists.org/lists/nmap-dev/2004/Apr-Jun/0024.html .
+ Actually I do indirectly use that for Windows. I wonder if they work
+ for UNIX?
+
+o perhaps each 'match' line in nmap-service-probes should have a
+ maximum lines, bytes, and/or time by which a response should be
+ available. Once that much time (or many bytes or lines) have passed,
+ that match can be considered 'failed' and ignored in subsequent runs.
+ Once all matches are considered failed, that probe is done. This
+ could be a useful optimization and is arguably better than the less
+ granular 'totalwaitms'. Or I could just have a simple function that
+ looks at whether a given regex could possibly match something
+ starting with the received data (not too hard since almost all of
+ the current regexes are anchored). But before doing this, I should
+ look long and hard at how many of the probes have every match
+ capable of doing this. In particular, many of the softmatch lines
+ don't offer many chars anchored at the front.
+
+o Separate nbase into its own Windows library in the same way as Andy did
+ with iphlpapi .
+
+o Nmap / Nmap-hackers FAQ
+
+o random tip database
+
diff --git a/todo/nping.txt b/todo/nping.txt
new file mode 100644
index 0000000..c1130cf
--- /dev/null
+++ b/todo/nping.txt
@@ -0,0 +1,799 @@
+/*****************************************************************************
+ * *
+ * o *
+ * o *
+ * o *
+ * o o *
+ * o o *
+ * o o *
+ * o o o *
+ * o o o *
+ * 888b 888 o o o *
+ * 8888b 888 o o o *
+ * 888Y88 888 o o o *
+ * 888Y88b 888 o *
+ * 888 Y88b888 o *
+ * 888 Y88888 *
+ * 888 Y8888 *
+ * 888 Y888 *
+ * *
+ * --[NPING TO-DO LIST]-- *
+ * *
+ *****************************************************************************/
+
+ This file contains Nping's to-do list. Items are listed in order of priority
+ (high priority items are listed first). Feel free to work on any of the items
+ on the list. However, if you'd like to work on something that is not trivial
+ to implement you may want to send a message to the nmap-dev list before you
+ start so other developers can see what you are planning to do. Make sure you
+ explain exactly what you are trying to fix/implement and how you are planning
+ to do it. It's always better to discuss bugfixes and new feature additions in
+ advance because they may actually have bigger implications than you think and
+ you may not get your patch accepted.
+
+ Please keep in mind that contributed code must:
+ * Be written in C++.
+ * Include comments so anyone can understand immediately what it does.
+ * Work on Linux, Mac OS and MS Windows. It's OK if you have not tested
+ the code in all those platforms, but at least keep portability in mind when
+ you write it and include a list of systems you've tested it on along with
+ your patch.
+
+ Questions, comments and patches should be sent to the Nmap development
+ mailing list (nmap-dev). To suscribe:
+ <https://nmap.org/mailman/listinfo/dev>
+
+
+/*****************************************************************************
+ * Things that have NOT been done yet *
+ *****************************************************************************/
+
+* Improve IPv6 support. Currently it doesn't work well. The situation should be
+ analyzed in detail because right now Nping has code to send packets at raw
+ transport level (letting the OS craft the IPv6 header), and at raw ethernet
+ level. None of them seems to work well, though.
+
+* Investigate an IPv6-related core dump reported by Vasiliy Kulikov.
+ More info: http://seclists.org/nmap-dev/2011/q3/567
+
+* Consider using Nmap's proto-dependant payloads for UDP packets. According
+ to David's tests, better results are obtained when sending UDP probes with a
+ payload specific to the protocol.
+
+* Consider adding the possibility to see the RTT in the RECV line. Something
+ similar to the way the traditional ping tool prints the RTT (time=XXX ms)
+
+ $ ping nmap.org
+ PING nmap.org (173.255.243.189) 56(84) bytes of data.
+ 64 bytes from nmap.org (173.255.243.189): icmp_req=1 ttl=48 time=169 ms
+ 64 bytes from nmap.org (173.255.243.189): icmp_req=2 ttl=48 time=177 ms
+ 64 bytes from nmap.org (173.255.243.189): icmp_req=3 ttl=48 time=179 ms
+ ^C
+ --- nmap.org ping statistics ---
+ 3 packets transmitted, 3 received, 0% packet loss, time 2000ms
+ rtt min/avg/max/mdev = 169.097/175.137/179.152/4.347 ms
+
+
+ This was requested by Jacek Wielemborek. More info:
+ http://seclists.org/nmap-dev/2013/q3/533
+
+* Currently, Nping determines the maximum number of open descriptors
+ (in TCP connect and UDP unprivileged modes), from the value returned
+ by libnetutil::get_max_open_descriptors(). However, it is often the
+ case that such function returns a value higher than FD_SETSIZE, which
+ is the maximum number of descriptors that select(2) can handle.
+ Currently Nsock uses select(2) so we have to limit the number of
+ descriptor to FD_SETSIZE, and not to the value returned bu
+ get_max_open_descriptors(). However, Henri Doreau is working on a new
+ nsock-engines branch which will provide Nsock engines based on
+ better I/O syscalls like poll() and epoll(). I've asked Henri if he
+ could implement a function in Nsock that provides the maximum number
+ of descriptors that can be handled at the same time, based on the
+ nsock engine being used. So, if that function gets implemented and
+ his nsock-engines branch merged into trunk, we should consider
+ updating Nping's code to use it.
+ More info here:
+ http://seclists.org/nmap-dev/2011/q4/550
+
+* A few ideas for the Echo protocol:
+ - Add an authenticated NEP_BYE message, so session termination is explicit
+ and both ends can determine if the session was ended because the other end
+ requested it or if it was due to some error at the network or transport
+ layer. Suggested by David.
+
+ - Add examples for encryption and hmac to the RFC. This would help in
+ debugging implementations. Suggested by Toni Ruottu.
+
+ - RFC. Improve description of how the IVs work. Suggested by Toni Ruottu.
+
+ - RFC. Improve description of encryptionless sessions. Suggested by Toni
+ Ruottu.
+
+ - Currently, the echo server zeroes any application layer data before
+ transmission in a NEP_ECHO message. This minimizes the impact of
+ errors in the server's packet matching engine or malicious attacks that
+ attempt to trick the server into echoing packets that do not belong to
+ a particular user. This works well but in the future, if one day we
+ create a NEPv2 specification, we may want to consider extending NEP_ECHO
+ packets to allow stripped-packet transport. This is, to allow echo servers
+ to remove application layer data before transmission, and include
+ additional information in the NEP_ECHO message so clients can determine
+ that the payload part was stripped and how long was it.
+
+ - Consider making the echo server bind to all IPv4 AND IPv6 interfaces.
+
+ - Add a description of the security implications of running a public echo
+ server (failures in the packet matching algorithm, etc), to either the
+ RFC or the man page. Suggested by Toni Ruottu.
+
+ - Test the new --safe-payloads option with a packet fuzzer to make sure
+ the packet parser behaves correctly.
+
+* When running Nping echo client with the --no-capture parameter, the last
+ packet's CAPT line is not displayed.
+
+ nping --ec public echo.nmap.org -p90 --tcp --count 1 --no-capture
+
+ luis@Aberdeen:~$ sudo nping --ec public echo.nmap.org -p90-92 --tcp --count 1 --no-capture
+
+ Starting Nping 0.5.52.IPv6.Beta2 ( https://nmap.org/nping ) at 2011-07-05 12:53 CEST
+ SENT (7.3302s) TCP 163.117.203.253:18554 > 74.207.244.221:90 S ttl=64
+ CAPT (7.4625s) TCP 163.117.203.253:18554 > 74.207.244.221:90 S ttl=54
+ SENT (8.3309s) TCP 163.117.203.253:18554 > 74.207.244.221:91 S ttl=64
+ CAPT (8.4429s) TCP 163.117.203.253:18554 > 74.207.244.221:91 S ttl=54
+ SENT (9.3310s) TCP 163.117.203.253:18554 > 74.207.244.221:92 S ttl=64
+
+ Max rtt: N/A | Min rtt: N/A | Avg rtt: N/A
+ Raw packets sent: 3 (120B) | Rcvd: 0 (0B) | Lost: 3 (100.00%)| Echoed: 2 (80B)
+ Tx time: 2.00181s | Tx bytes/s: 59.95 | Tx pkts/s: 1.50
+ Rx time: 2.00193s | Rx bytes/s: 0.00 | Rx pkts/s: 0.00
+ Nping done: 1 IP address pinged in 9.33 seconds
+
+* Sometimes Nping displays a couple of error messages (related to cleanup of
+ Nsock events), even though everything went fine.
+
+ luis@Aberdeen:~$ sudo nping --ec public echo.nmap.org -p90 --tcp --count 1
+
+ Starting Nping 0.5.52.IPv6.Beta2 ( https://nmap.org/nping ) at 2011-07-05 12:51 CEST
+ SENT (1.8965s) TCP 163.117.203.253:64288 > 74.207.244.221:90 S ttl=64
+ CAPT (2.0293s) TCP 163.117.203.253:64288 > 74.207.244.221:90 S ttl=54
+ RCVD (2.1233s) TCP 74.207.244.221:90 > 163.117.203.253:64288 RA ttl=51
+ nping_event_handler(): READ-PCAP killed: Resource temporarily unavailable
+ nping_event_handler(): TIMER killed: Resource temporarily unavailable
+
+ Max rtt: 226.762ms | Min rtt: 226.762ms | Avg rtt: 226.762ms
+ Raw packets sent: 1 (40B) | Rcvd: 1 (40B) | Lost: 0 (0.00%)| Echoed: 1 (40B)
+ Tx time: 0.00136s | Tx bytes/s: 29411.76 | Tx pkts/s: 735.29
+ Rx time: 1.00082s | Rx bytes/s: 39.97 | Rx pkts/s: 1.00
+ Nping done: 1 IP address pinged in 2.93 seconds
+
+* Investigate about warning on old version of gcc like g++ 4.1.2 20080704
+ (Red Hat 4.1.2-48). No warnings are shown on newer version but it would be
+ nice to get rid of them if possible. There are some of them:
+
+ ARPHeader.h:169: warning: ‘class ARPHeader’ has virtual functions but
+ non-virtual destructor
+ RawData.h:99: warning: ‘class RawData’ has virtual functions but
+ non-virtual destructor
+
+* Decide more on rDNS
+ - Do we want to rDNS resolve all target IPs? If so, where should we
+ show the name? At the final report (even when just one host
+ scanned, which omits that line now)? In the individual packet
+ trace lines? When a CNAME (or a name which forward resolves but
+ does the IP doesn't reverse resolve) is specified on the command
+ line, should we use that version, or the official rDNS, if any?
+ - Some more discussion on this topic on nmap-dev may be warranted.
+
+* Improve output for negative verbosity levels. Currently, one can't
+ even tell how many hosts replied, just how many responses were
+ received, which could be all from the same host. If there is only
+ one target, then the current behavior is fine. However, when pinging
+ more targets, we should be able to provide a better output; at least
+ how many hosts were alive. This was suggested by Dan Farmer.
+
+* Consider adding more examples of setting fields/payloads to the man
+ page. This was suggested by Dan Farmer.
+
+* Consider adding support for XML output.
+
+* From: David Lam <david@thedavid.net>, "Some general questions about
+ Nping/Ncat"
+
+ In TCP traceroute mode, would it be possible to ask Nping to
+ stop once it gets an SYN-ACK response back from the destination host rather
+ than continuously hitting the host until the max TTL?
+
+* Make broadcast ping work. Currently the following command does not
+ show any captured packets:
+ nping 192.168.0.255 --dest-mac ff:ff:ff:ff:ff:ff -c 1
+ The cause is probably the BPF filter, which only allows replies from
+ 192.168.0.255.
+ Also, look into official multicast addresses like 224.0.0.1. Can we
+ received replies to that probe?
+
+
+* Do some performance testing.
+ Fyodor:
+ <<Nping should be able to send packets quickly, at least comparable to
+ "ping -f" and hping. If it can't send as many packets per second as those,
+ then it warrants looking into whym figuring out what the bottlenecks are.
+ It would be good to compare nping with other tools such as hping in
+ terms of how high the values of packets per second can get and still
+ work reliably.>>
+
+* Stats for ARP packets.
+
+* Do more testing on Mac
+
+* Support pre defined probe rates: --fast, --faster, --flood, --slow,
+ --slower, --paranoid...
+
+* Think about --establish feature, which uses raw packets to establish
+ a connection and can then send data on the connected stream (Luis
+ already has a proof-of-concept implementation).
+
+* Make privileged and unprivileged TCP/UDP mode specification consistent.
+
+> - User is unprivileged and did not supply mode: --> Use TCP-Connect
+> - User is unprivileged and supplied --tcp --> Use TCP-Connect
+> - User is unprivileged and supplied --upd --> User UDP unprivileged
+> - User is root and did not supply mode --> Use ICMP Echo
+> - User is root and supplied --tcp --> Use raw sockets TCP
+> - User is root and supplied --udp --> User raw sockets UDP
+> - User is root and wants to use TCP-Connect --> User needs to either
+> pass --tcp-connect or --unprivileged
+> - User is root and want unprivileged UDP --> User needs to pass
+> --unprivileged or --udp-XXXXX (any suggestions?. --udp-sendto() may not
+> be the best idea because when we use raw sockets we also use sendto() to
+> transmit the data).
+
+* Support reverse DNS resolution in --traceroute
+
+* Implement TCP options
+
+* Implement hping-like ability to change the port/ttl using the keyboard
+ during a scan.
+
+* Disable ARP resolution when --source-mac is specified.
+
+* Implement --data-file option. What should we do if file is big? Read the
+ first X bytes? Send consecutive chunks?
+
+* Implement ICMP address mask
+
+* Implement entire ICMP Traceroute message opts.
+
+* Research on default IP Identification value. Kernel does not seem to like
+ value 0 because when set to zero, kernel changes it to some other value. When
+ we set it to something !=0, the kernel leaves our value untouched.
+
+* At some point in the future, implement weird ICMP Types. I think this would
+ let us make a difference to the rest of pings and packet creation tools
+ because anyone wanting to send weirds packes would have to download our
+ Nping ;-)
+ ( http://www.iana.org/assignments/icmp-parameters )
+ 6 Alternate Host Address [JBP]
+ 31 Datagram Conversion Error [RFC1475]
+ 32 Mobile Host Redirect [David Johnson]
+ 33 IPv6 Where-Are-You [Bill Simpson]
+ 34 IPv6 I-Am-Here [Bill Simpson]
+ 35 Mobile Registration Request [Bill Simpson]
+ 36 Mobile Registration Reply [Bill Simpson]
+ 39 SKIP [Markson]
+ 40 Photuris [RFC2521]
+
+* Implement checks in function that handles received packets:
+ Fyodor:
+ <<You can't assume that the filter always works right, so you do need to
+ validate the information anyway. For example, on windows in some cases
+ we have to change the filter to "" because it doesn't work otherwise
+ so, in actuality, I often end up with rather broad pcap filters and then
+ do the checking by hand, but tightening the pcap filter can improve
+ performance a bit.>>
+
+* Implement "-iL inputfilename (Input from list) " and the case where "-" is
+ supplied and target specs need to be read from stdin.
+
+* Consider adding option to allow sending NO packets but act as a
+ simple sniffer. Users could use --bpf-filter to specify a
+ tcpdump-like filter and get every receive packet printed to
+ stdout. Maybe with "-c 0"? "-c none"? We need to have some flag in
+ NpingOps so we don't terminate Nping but wait undefinitely.
+
+* At some point we should support nmap-like MAC specification.
+
+* When implementing IPv6, check MAX_TCP_PAYLOAD_LEN constant and method
+ TCPHeader::setSum(). Because with IPv6 the max payload length should be 20
+ bytes less than with the IPv4 header.
+
+* When using payloads, take into account that the IP and TCP headers may
+ contain options and therefore, the maximum payload len should be
+ 65535 - 20(ip header) - 40 (ip options) -20(tcp header) -20(tcp options);
+
+* Make sure randomnly generated checksums in IPv6-TCP/UDP are in fact invalid
+ and don't match the correct checksum.
+
+* Fyodor:
+ <<in some cases it might be nice to have an option which sends all
+ probes (all ports to all hosts) at the same time.>>
+
+* ARP mode does not support payload specification. However, users may
+ want to do things like appending null bytes at the end of an ARP
+ packet to test some device behaviour, etc. Adding support for
+ payload to this mode is really trivial, would make the payload spec
+ more consistent with the rest of the modes, and may be a nice to have
+ feature.
+
+* [EM] For CAPT packets, decide if we want to print the full info or
+ just the fields that have changed in transit (or both). Note that
+ printing differences would be complicated by the fact that nping
+ doesn't currently associate captured packets with the original send.
+
+* Decide if we want to allow things like "1074628148" or "0x400d8634" to
+ be treated as valid IP addresses.
+
+* Check out if --ip-options "RTUS 1.1.1.1 2.2.2.2" makes sense. It now
+ fails.
+
+* It may be nice to let users set the IP header lenght field. Maybe they
+ want to stress tcp/stacks with this.
+
+* Investigate on ICMP preference levels. It's not clear whether there is
+ a standard encoding or not. The logic that parses this in Nping needs
+ to be reviewed.
+
+* Split up libnetutil.cc into different source files.
+
+* Investigate on nping's version of devname2ipaddr. Think about side
+ effects on using that in Nmap.
+
+* Consider adding multi-packet support.
+ o Example: tell nping to send 4 tcp packets, 5 icmp packets and 3 udp packets
+
+* Consider adding RFC-style output for send/recv packets.
+
+* Consider adding more detailed stats for the Echo Mode.
+
+* [EM] Handle DLT types. Currently the server always sets the null DLT value
+ that indicates that no data link header is included.
+
+/*****************************************************************************
+ * Things that have been solved already *
+ *****************************************************************************/
+
+[DONE] Add default target port for TCP-Connect and TCP modes :: Port 80
+
+[DONE] Add default target port for UDP mode :: Port 40125
+
+[DONE] Add default UDP Source port: 53
+ JUSTIFICATION: From David's EffectivenessOfPingProbes
+ http://www.bamsoftware.com/wiki/Nmap/EffectivenessOfPingProbes
+ "The best individual UDP probes are still those to a random high port,
+ with a source port of 53 and a non-empty payload. Even without the source
+ port and payload, the ports 40125 and 40126 that I picked out of the air
+ are better choices than the current default of 31338, finding around 400
+ additional hosts."
+
+[DONE] Change resolution for the inter-ping delay. (Fyodor: btw, usleep() will
+ probably do the trick for you as it let's you sleep with microsecond
+ precision)
+
+[DONE] Use int send_ip_packet(int sd, struct eth_nfo *eth, u8 *packet, unsigned int
+ packetlen) instead of ip_open();
+
+[DONE] Add protocol to BPF filterstring because It is possible that when in TCP mode
+ a UDP packet destined to the TCP source, arrives to the net iface and gets
+ printed.
+
+[DONE] Implement multiple port specification.
+
+[DONE] Implement ICMP router advertisement entries
+
+[DONE] Default probe mode: ICMP echo
+
+[DONE] Test ICMPv4Header::addRouterAdEntry() and check entries are being added
+ correctly.
+
+[DONE] Determine source IP address automatically
+
+[DONE] Determine network interface to be used for packet capture automatically
+
+[DONE] Add support for cached DNS requests
+
+[DONE] Start user documentation (mainly man page)
+
+[DONE] Change output to include timing information
+
+[DONE] Implement controls in payload options parsing to prevent specifying lengths
+ that cannot be carried by a single TCP/UDP packet.
+
+[DONE] Start implementing unprivileged UDP pings.
+
+[DONE] When sending ICMP packets, checksum is not being computed correcly if
+ --data-length, and options like that, are specified.
+
+[DONE] Find a bug that under some circumstances produces a segfault. It is probably
+ related to the way option -e is being handled.
+
+[DONE] Fix a bug in option "-e iface" that results on IP 2.0.0.0 being used as a
+ source address.
+
+[DONE] Update --help display to include new ICMP flags. Check also commandline syntax
+ docs.
+
+[DONE] Use nsock approach instead of threads.
+
+[DONE] Finish ARP/RARP support.
+
+[DONE] Change doc for option --count. We don't stop after N probes, we stop after
+ N rounds.
+
+[DONE] Ask Fyodor what tool is used to convert from nmap-man.xml to nmap.1
+
+[DONE] Check all outPrint()s and outError()s to ensure they specify the correct
+ verbosity/debug level.
+
+[DONE] Document format specified in ArgParser::atoICMPType().
+
+[DONE] Document format specified in ArgParser::atoICMPCode().
+
+[DONE] Finish implementing unprivileged UDP pings.
+
+[DONE] Finish Ethernet frame creation.
+
+[DONE] Find a way to convert the nping.xml into man page.
+
+[DONE] Check what happens if payload is specified and we are not sending TCP/UDP
+ but ICMP or other proto packets. [Sometimes it may not make sense to include
+ payloads (e.g. ARP) but we still allow it just in case users want to play
+ around].
+
+[DONE] Ask Fyodor whether we want to display elapsed time (like nmap) or we prefer to
+ display rtt time as other ping utilities do. [This is probably fine for now]
+
+[DONE] Fix the warnings produced by Fyodor's gcc.
+ +---------------+
+ NpingTargets.cc: In member function ‘int NpingTargets::processSpecs()’:
+ NpingTargets.cc:315: warning: comparison between signed and unsigned integer expressions
+ NpingTargets.cc: In member function ‘NpingTarget* NpingTargets::getNextTarget()’:
+ NpingTargets.cc:333: warning: comparison between signed and unsigned integer expressions
+ +---------------+
+ In file included from /usr/include/string.h:640,
+ from nbase/nbase.h:158,
+ from nping.h:107,
+ from utils.cc:95:
+ In function ‘void* memset(void*, int, size_t)’,
+ inlined from ‘int getNetworkInterfaceName(sockaddr_storage*, char*)’ at utils.cc:689:
+ /usr/include/bits/string3.h:85: warning: call to void* __builtin___memset_chk(void*, int, long unsigned int, long unsigned int) will always overflow destination buffer
+ +---------------+
+
+
+[DONE] Redesign verbosity levels:
+ * Put verbosity levels 2 into level 1
+ * Use level 2 for error.
+ * Use level 3 to print everything but not sent/rcv packets.
+ * Level 4 the usual
+
+[DONE] Add stats at the end of nping execution.
+
+[DONE] Add options to disable viewing of sent packets.
+
+[DONE] Add option to to disable packet capture.
+
+[DONE] Add a section to the man page explaining how we iterate over targets,
+ ports, etc.
+
+[DONE] Beta-testing email to the list.
+
+[DONE] Change default round count to 5.
+
+[DONE] Fix a segfault detected by Fyodor in trg=o.targets.findTarget(...).
+
+[DONE] Send an email to the list telling about the nping.exe file.
+
+[DONE] Support CTRL-C statistics.
+
+[DONE] Change "solution" file in mswin32/nmap.sln to nping.sln
+
+[DONE] In man page and -h: move Ethernet section so it appears after network
+ layer info.
+
+[DONE] Make rx time more accurate taking into account that we wait for a bit after
+ the last probe is sent.
+
+[DONE] Fix bug: add ICMP dest unreachable, etc to the BPF filter so we can get
+ icmp error messages when TTLs expire, etc.
+
+[DONE] Disable all ethernet related code when sendEth is false.
+
+[DONE] Finish porting Nping to Windows.
+
+[DONE] Find an OS X box to test Nping.
+
+[DONE] Reorganize verbosity levels (again ;-) [-3, +3].
+
+[DONE] Finish documentation for options --source-mac and --dest-mac
+
+[DONE] Make sure --ether-type supports specifying types in hex.
+
+[DONE] Implement verbosity level 3: in this level, sent and recv packets are
+ hexdumped to stdout.
+
+[DONE] Write and check in nping/index.html web site
+ - Include SVN checkout/install instructions
+ - include tarballs when available
+
+[DONE] Create Windows installer (maybe can copy a lot of stuff from what
+ Ithilgore has done with Ncrack)
+
+[DONE] Create Nping release tarball for UNIX systems
+
+[DONE] Release Nping 0.1BETA2
+
+[DONE] Man page should say Nping is currently in Alpha stage.
+
+[DONE] Support -vvv, -qqq and -ddd syntax. [Requested by Dirk Loss]
+
+[DONE] Create Mac OS X installer (also can probably copy a lot of stuff
+ from what Ithilgore has done with Ncrack. David can usually help
+ with installer building).
+
+[DONE] Move nping to /nping in SVN rather than being in nmap-exp
+
+[DONE] Set up automatic conversion from nping XML man page to HTML for
+ https://nmap.org/nping/man.html [Fyodor working on this]
+
+[DONE] Include signature files in new releases. [Requested by Henri Salo]
+[DONE] It would be nice to have Bzip2 packages. [Requested by Henri Salo]
+ (These last two don't make sense anymore as Nping is now distributed
+ with Nmap).
+
+[DONE] Do small fix in nmap's send_ip_packet_sd()
+ - res = Sendto("send_ip_packet", sd, packet, packetlen, 0,
+ + res = Sendto("send_ip_packet_sd", sd, packet, packetlen, 0,
+
+[DONE] Correct BPF filter specs, to make the condition about the source
+ address apply everywhere.
+
+[DONE] Fix possible bug in BPF filter specification. More details in
+ http://seclists.org/nmap-dev/2010/q2/252
+
+[DONE] Work on nping&nmap code merge.
+
+[DONE] For options that take numbers we need to allow users to specify them
+ also in hex with the format 0xNNNN...
+
+[DONE] Replace this pattern:
+ if ( isNumber_u32(optarg) ){
+ u32 aux32 = strtoul( optarg, NULL, 10);
+ ...
+ }
+ with a function that checks for syntax and returns the value (i.e., a wrapper
+ around strtoul). There is nowhere that isNumber_u* is called without it being
+ immediately followed by a strtoul, outside of utils.cc.
+
+[DONE] Bug in --icmp-advert-entry. Specified IPs are being set in host byte
+ order instead if in network byte order.
+
+[DONE] Investigate why ARP replies are not being received. Wireshark shows
+ replies but they don't get captured by Nping. The bpf filter looks
+ ok: "arp and arp[6]==0x00 and arp[7]==0x02"
+
+[DONE] Investigate into this:
+ sudo nping --icmp scanme.nmap.org -vvv -d1 --icmp-type ra --icmp-advert-entry 256.257.258.259,222
+ Invalid Router Advertising Entry specification: Unable to resolve 6628128
+ Apparently the call to outFatal() is specifying %d instead of %s, but
+ that's not being detected properly by the compiler, because we don't
+ get a warning. We have to do something like this:
+ void fatal(const char *fmt, ...)
+ __attribute__ ((noreturn))
+ __attribute__ ((format (printf, 1, 2)));
+ TODO: Look at the documentation to see what the numbers mean.
+ Probably one of the is the index of the format argument, and the
+ other is where the varargs start.
+
+[DONE] Fix division by zero exception:
+ sudo nping --icmp scanme.nmap.org -vvv -d1 --icmp-type echo --rate 0
+ ./test_nping.sh: line 83: 11690 Floating point exception"$@"
+
+[DONE] Fix little problem in TIMING_5. We need to detect the bogus time
+ before we actually pass the value to NpingOps. Nping is giving an
+ error but the bogus input is getting to far.
+
+[DONE] Document that badsum-ip may not always work because the kernel may
+ correct the sum.
+
+[DONE] Change overloaded functions in libnetutil that were refactored to
+ make them compile in C. Go back to the overloaded version if possible.
+
+[DONE] Move grab_next_host_spec() and pals to netutil.
+
+[DONE] Control the case when user passes "--mtu 0". An assertion fails but
+ Nping should print a nicer message.
+
+[DONE] Improve error message for --mtu. We should probably allow mtu's bigger
+ than 2^16 but take that as a "dont fragment" request. Also, make
+ "rand" produce only valid MTUs (multiple of 8, etc).
+
+[DONE] When passing "--tcp-flags 0x100" the error is not very accurate.
+ This is because parser_u8() fails and then Nping tries to resolve the
+ value letter by letter. Maybe we can parse_u32() it, and then check
+ if n<255 and print a better error message.
+
+[DONE] Document what happens with the IP header length when user wants to
+ add uneven bytes of IP options. We are truncating the result, because
+ the header length is expressed in 32 bit words.
+
+[DONE] Check if there is any problem with -e "". Maybe we shouldn't let users
+ supply a NULL name, but make them use the "any" specifier. Add doc
+ about this and update the test description (MISC_12).
+
+[DONE] Update documentation for option --delay, including that now, time
+ specification as float numbers is supported (eg: --delay 0.1 meaning 100ms)
+
+[DONE] Change info about TODO file in https://nmap.org/nping web page.
+ - If you wish to contribute code to Nping there is a TO-DO list you can have
+ - a look at (file "TODO" in the source package).
+ + If you wish to contribute code to Nping there is a TO-DO list you can have
+ + a look at (file "todo/nping.txt" in nmap's source package).
+
+[DONE] Make sure randomnly generated checksums are in fact invalid and don't match
+ the correct checksum. There is a 1/65535 chance of this happening.
+
+[DONE] After merging nmap-dedup, change send_frag_ip_packet() to take "u32 mtu"
+ and fix the printf below to use "%u" instead of "%i".
+
+[DONE] [EM] Update EchoProtoRFC.txt and any of the other design files as
+ appropriate and send to nmap-dev for comments
+
+[DONE] [EM] Pick a default port number
+
+[DONE] [EM] Make a mockup of the desired standard output in a regular echo mode
+ execution, like nping -c 2 --tcp --flags SYN -p 80 scanme.nmap.org (let's
+ assume there are some differences found, like a NAT is in place)
+ o A key aspect of this task is determining what diffs are going
+ to look like.
+
+[DONE] [EM] Things to decide on:
+ o Decide on packet specifiers that can be passed to the server so it
+ can recognize packets sent by the client even if a number of headers
+ have changed and pass them back. (see Fyodor/Luis IM discussion logs
+ from 6/28/10).
+
+[DONE] [EM] Improve client error handling. Currently it doesn't behave well when
+ the server crashes.
+
+[DONE] [EM] Make the client timeout if the server does not send data during
+ handshake. Currently the client waits forever.
+
+[DONE] [EM] Make the server detect when a client disconnects and delete its context
+ data.
+
+[DONE] [EM] Get rid of some messages that are currently displayed in the client.
+ Print them only if debugging level is high enough.
+
+[DONE] [EM] Make sure -h help screen includes info about the echo mode.
+
+[DONE] [EM] Add echo mode to the man page.
+
+[DONE] [EM] Add received echoed packet to the final statistics.
+
+[DONE] [EM] Multi-client support
+
+[DONE] [EM] Delay RECV message printing so the CAPT messages are shown in order.
+
+[DONE] [EM] Use NEP_QUIT only if necessary, just close connection if possible.
+
+[DONE] [EM] Implement crypto
+
+[DONE] [EM] Consider whether the CAPT line should (or should have an
+ option to) display the time based on capture time from the server.
+ Obviously this can be problematic because not all machines run
+ ntpd. One option is to just make it an option so that people should
+ only use it if both the client and server are running ntpd. Luis is
+ adding a precision timestamp to NEP_ECHO packets so we could easily
+ add it in the future. Another approach would be to do NTP-style
+ handshaking to compute time offsets between the two machines during
+ the echo side-channel handshaking. Then the client could remember
+ how far off it is. A third approach is to guess about the CAPT time
+ that it was 1/2 the time between packet send and when we received
+ the NEP_ECHO back notifying us of receipt.
+ NOTE: We finally decided to take the third approach. CAPT_time=RTT/2.
+
+[DONE] [EM] Consider whether we should delay RCVD packet printing
+ slightly so that CAPT packets received just slightly afterward could
+ be printed before the RCVD. This might make the most sense if we do
+ the previous feature where we show the time that a packet was
+ actually captured by echo server. If we did it in normal cases, it
+ might make it easier to compare SENT and CAPT packets, but would
+ also be a bit strange to see the timeline out-of-order.
+
+[DONE] Fix Windows rtt values. Right now Nsock does not seem to be giving
+ the callback at the proper time, or something.
+
+[DONE] Add --no-crypto to -h output.
+
+[DONE] Make sure nping does not allow generating packets with tcp src port or
+ tcp dst port 9929 (or --echo-port N, if that is set), because 1) the
+ echo server does not capture those packets and 2) to avoid messing up the
+ established side-channel tcp connection.
+
+[DONE] Add support for custom IP binding: if user supplies -S then
+ the echo side-channel connection and connections in TCP-Connect mode should be
+ established from that IP. This includes the echo server binding to that IP.
+
+[DONE] Make nping issue a warning when user supplies a payload in TCP-Connect
+ mode.
+
+[DONE] [EM] Echo server should print which interface is using to capture packets.
+
+[DONE] In some cases, when using nping through a VPN connection, nsi_pcap_linktype()
+ returns something different to DLT_EN10MB, and Nping fatals. Investigate
+ why this happens to nping and is not a problem for Nmap. Also, determine
+ why this doesn't happen all the time. What does it change between these
+ two?: sudo nping --udp 1.1.1.1 -g 999 -p998
+ sudo nping --udp 1.1.1.1 -g 999 -p999
+ The first one works, and the other one fatals with the "Currently only
+ Ethernet is supported." (error message @ nping.cc:1717).
+ - Note this also happens when Fyodor uses Nping tethering through
+ his cell phone (ppp0)
+
+[DONE] [EM] Make the server stop capturing packets when all connected clients
+ finish their session.
+
+[DONE] [EM] Some things to keep in mind for the implementation and to update
+ our design docs accordingly:
+ o Implement different "modes" for the server: complete access,
+ one-time-access, and restricted.
+
+[DONE] Do more testing on MS Windows.
+
+[DONE] [EM] Investigate why the echo server does not send NEP_ECHO messages when the
+ client sends probes at a very high rate, like in :
+ ./nping -c 1000 --rate 1000 --echo-client "pass" --icmp -v echo.nmap.org
+
+[DONE] [EM] Add echo mode to the man page
+
+
+[DONE] [EM] Do some extensive testing of the Echo mode once it is working
+ to try and flesh out any bugs before merging.
+
+[DONE] Make Nping call nsi_delete() on pcap IODs, IODs in TCP-Connect mode and maybe
+ in IODs of other modes. See http://seclists.org/nmap-dev/2010/q3/587
+
+[DONE] Fix bug that causes Nping to fail when sending UDP packets to a broadcast
+ address. More info: <http://seclists.org/nmap-dev/2010/q3/752>
+
+[DONE] When doing ICMP echo traceroute (with --traceroute), unless the user
+ supplies a custom round count (-c/--count), Nping only sends 5 packets
+ (default round count). This is usually not enough to reach hosts
+ on the internet. What should be the default behaviour? Stick with the
+ default round count of 5 or increment it when --traceroute is set?
+ - We should probably set -c 32 when --traceroute is specified,
+ unless user specifies their own -c explicitly.
+
+[DONE] Try to reduce the size of the internal buffer in the EchoHeader class.
+ Currenltly it allocates a big buffer that is able to hold the theoretical
+ maximum size of a NEP message (normal use does not require so much space).
+ When this is done, check if we still need to increase the stack size
+ in the project properties in Visual Studio.
+
+[DONE] [Fixed by Vasiliy Kulikov] When running Nping in ARP mode, hexdump of
+ ARP replies is not shown with -vvv, only for requests. Here's the output:
+
+sudo nping --arp 192.168.240.139 -vvv -d1
+
+Starting Nping 0.5.59BETA1 ( https://nmap.org/nping ) at 2011-07-11 12:32 CEST
+BPF-filter: arp and arp[6]==0x00 and arp[7]==0x02
+SENT (0.0562s) ARP who has 192.168.240.139? Tell 192.168.240.1
+0000 ff ff ff ff ff ff 00 50 56 c0 00 01 08 06 00 01 .......PV.......
+0010 08 00 06 04 00 01 00 50 56 c0 00 01 c0 a8 f0 01 .......PV.......
+0020 00 00 00 00 00 00 c0 a8 f0 8b ..........
+RCVD (0.0568s) ARP reply 192.168.240.139 is at 00:0C:29:E4:90:CD
+SENT (1.0580s) ARP who has 192.168.240.139? Tell 192.168.240.1
+0000 ff ff ff ff ff ff 00 50 56 c0 00 01 08 06 00 01 .......PV.......
+0010 08 00 06 04 00 01 00 50 56 c0 00 01 c0 a8 f0 01 .......PV.......
+0020 00 00 00 00 00 00 c0 a8 f0 8b ..........
+
+
diff --git a/todo/patrick.txt b/todo/patrick.txt
new file mode 100644
index 0000000..7508afd
--- /dev/null
+++ b/todo/patrick.txt
@@ -0,0 +1,77 @@
+===
+
+Currently working on:
+
+-- LPEG in NSE.
+
+-- HTTP Library in LPeg.
+
+===
+
+Maybe:
+
+-- NSE Debugger. Look at Diman's implementation:
+ http://seclists.org/nmap-dev/2008/q1/0228.html
+ http://www.keplerproject.org/remdebug/
+
+-- Review NSE Nsock Socket Allocation:
+ o Dynamically increase socket slots if nothing has been done
+ in the last ~5 seconds. Also decrease once traffic is working again.
+ This resolves any sort of socket deadlock.
+
+-- Deadlock identification and correction:
+ o Add detection for deadlocks and print which threads are involved.
+ o use above results to make a strategy for automatic deadlock resolution.
+
+-- Look into moving Packet Module to C.
+
+===
+
+Done:
+
+-- Review and Improve NSE Nsock Library.
+ o Move away from C pointer references and allocation over to Lua.
+ If a function ends in error, all the userdata will be collected.
+ We would otherwise need to use pcalls everywhere to clean up
+ and free malloc()'d memory.
+ o Use thread calling nsock_loop (or currently running thread)
+ for restoring waiting threads to the running queue.
+ Making a function call on a yielded thread is a hack and
+ could cause problems in the future.
+ o Get rid of the static nsock_pool and use a dynamically allocated
+ structure on a per-host-group basis.
+ o Prepare for Lua 5.2 --> Change to real errors.
+
+-- Update NSE Book Implementation Section.
+
+-- Added boolean operator patch.
+
+-- Update NSE --script section (book) to include Boolean operators.
+
+-- Fix ceil for runlevels.
+
+-- Solve Brandon's Segfault for thread's sockets and close them when
+ the thread ends.
+
+-- Change the error on finding the name of a nonexistent file in script.db
+ into a non-fatal warning.
+
+-- Correct nsock_connect to unlock the socket slot if the connection fails.
+
+-- Remove packet.hextobin and packet.bintohex. Fix scripts that used them
+ to instead use bin.(un)pack.
+
+-- Commit --script-args patch and update the relevant section in the book.
+
+-- Deadlock identification and correction:
+ o Release mutexes upon script death.
+
+-- Review NSE Nsock Socket Allocation:
+ o Release socket locks on connection failure or timeout.
+ o Track active sockets in the nsock library and don't rely on
+ garbage collection for reallocation.
+
+-- HTTP Caching:
+ o Add ability to use a proxy to http.lua.
+ o Test http.lua performance using local caching proxy.
+ o Implement a cache in http.lua.
diff --git a/todo/paulino.calderon.txt b/todo/paulino.calderon.txt
new file mode 100644
index 0000000..6f88523
--- /dev/null
+++ b/todo/paulino.calderon.txt
@@ -0,0 +1,4 @@
+TODO:
+
+-Update wiki page.
+-Fix: http-enum does not work on windows. UNIX paths are hardcoded into the script. It also fails when running from a directory with spaces in the name. \ No newline at end of file
diff --git a/todo/sctp.txt b/todo/sctp.txt
new file mode 100644
index 0000000..55bf042
--- /dev/null
+++ b/todo/sctp.txt
@@ -0,0 +1,49 @@
+TODO.sctp $Id$ -*-text-*-
+
+o Further investigate SCTP functionality, as some people reported
+ problems (see this thread:
+ http://seclists.org/nmap-dev/2009/q2/0669.html)
+
+o Add support for UDP encapsulated SCTP (9899/udp).
+ Basically just wrap the SCTP packets into a UDP packet.
+ Think about how to add support for this to libdnet first.
+ See this Internet Draft by Michael Tuexen for the specs:
+ http://tools.ietf.org/html/draft-tuexen-sctp-udp-encaps
+ This is actually quite a challenging task due to the
+ current architecture of the scan engine. How to best
+ differentiate a UDP packet related to a UDP scan from a
+ UDP wrapped SCTP packet? How to unpack the UDP wrapped
+ SCTP packet in order not to duplicate a lot of code?
+ A good solution will be non-trivial.
+
+o Verify ICMP response handling for SCTP. Make sure all
+ ICMP types are handled in an optimal way (esp. destination
+ unreachable: protocol unreachable).
+
+o Consider removing 9899/sctp from the default port list.
+ 9899/udp is used for UDP encapsulated SCTP. One reason
+ to keep 9899/sctp is likely misconfigurations.
+
+o Investigate whether it makes sense to store scan state in
+ the itag/itsn fields for INIT scans.
+
+o Investigate the suitability of other SCTP chunks for port
+ scanning and implement more scan types if they turn out to
+ be worthwhile. One unverified idea is to experiment with
+ undefined chunk types and their first two magic bits to
+ provoke ERROR responses.
+
+o Add SCTP based service probing.
+
+o [Ncat] Consider implementing SCTP broker mode.
+
+o [NSE] Add SCTP support to NSE.
+
+o Investigate on differences between SCTP stacks and
+ implement SCTP based OS detection probes based on the
+ results. For example, BSD systems send the ASCII string
+ KAME-BSD in INIT-ACK chunks.
+
+o SCTP-enable scanme.nmap.org in order to make scanme.roe.ch
+ obsolete.
+
diff --git a/todo/shinnok.txt b/todo/shinnok.txt
new file mode 100644
index 0000000..b294e42
--- /dev/null
+++ b/todo/shinnok.txt
@@ -0,0 +1,150 @@
+In progress:
+============
+
+o We should offer partial results when a host
+ timeouts. I (Fyodor) have been against this in the past, but maybe
+ the value is sufficient to be worth the maintenance headaches. Many
+ users have asked for this. If we do implement this, we may want to
+ only print results for the COMPLETED phases (e.g. host discovery,
+ port scanning, version detection, traceroute, NSE, etc.) Trying to
+ print partial results of a port scan or NSE or the like might be a
+ pain. And if we print some results for a host which timeouts, we
+ should give a very clear warning that the results for that host are
+ incomplete. As an example, here is someone who hacked Nmap source
+ code to achieve this: http://seclists.org/pen-test/2010/Mar/108.
+ o Another benefit would be that it would allow us to clean
+ up/regularize the host output code. Right now there are I think
+ three places where a host's final output can be printed. If,
+ instead, that code just looked at what information was available and
+ printed that out only, we could potentially isolate it in just one
+ place.
+ o This also might let us provide a feature for skipping the rest of
+ an Nmap phase which is going too slowly (I think that has its own
+ Nmap TODO item).
+
+Hanging(waiting for further input, etc..):
+==========================================
+
+o Nmap *poor's man* test suite by expanding on what I already have in
+ /nmap-exp/shinnok/nmap-test-script.
+
+o NMAP reports different service results every so often with the same port.
+ http://seclists.org/nmap-dev/2011/q2/815
+
+o Review latest revision of Marek's ncat_proxy.patch - DONE
+ http://seclists.org/nmap-dev/2011/q2/573
+ o Commit approval pending
+
+Pending:
+========
+
+Pending (low priority):
+=======================
+
+o E-mail nmap-dev with GProfiles /ncrack
+ o Create new default username list:
+ http://seclists.org/nmap-dev/2010/q1/798
+ o Could be a SoC Ncrack task, though should prove useful for Nmap
+ too
+ o We probably want to support several lists. Like an admin/default
+ list like "root", "admin", "administrator", "web", "user", "test",
+ and also a general list which we obtain from spidering from
+ emails, etc.
+
+Potential:
+==========
+
+COMPLETED:
+==========
+
+o Add a --append-output option to ncat. [DONE - r25737]
+
+o libpcre/pcre.h - is cleared upon make distclean thus leaving the SVN
+ working directory dirty
+ http://seclists.org/nmap-dev/2011/q2/708
+
+o De-duplicate code by unifying ncat_broker.c and ncat_listen.c code paths,
+ either as a single file in ncat_listen.c or merge duplicate code in
+ ncat_listen.c and keep only broker specific code in ncat_broker.c(it it's a
+ lot of code, otherwise ncat_listen.c would do just fine).
+
+o Nmap should defer address parsing in arguments until it has read
+ through all the args. Otherwise you get an error if you use like -S
+ with an IPv6 address before you put -6 in the command line. You
+ get a similar problem (on David's IPv6 branch) if you do "-A -6"
+ (but "-6 -A works properly).
+
+o Delve into Lua and NSE and try to write some scripts to get the hang
+ of it and gain a better understanding of the NSE engine in Nmap.
+ o Written two NSE scripts, http-reverse-ip and http-google-email that
+ can be found in /nmap-exp/shinnok/nse.
+
+o E-mail nmap-dev with QtCreator usage steps for Nmap
+
+--
+o Ncat hangs on ssl -> REFACTORING
+ some refactoring left to be done to reduce code duplication
+ http://seclists.org/nmap-dev/2011/q2/842
+ o Commit current switch/ifdef refactoring patch.
+ o Research code deduplication even further.
+
+o Ncat chat (at least in ssl mode) no longer gives the banner greeting
+ when I connect. This worked in r23918, but not in r24185, which is
+ the one running on chat.nmap.org as of 6/20/11. Verify by running
+ "ncat --ssl -v chat.nmap.org"
+
+o Pending uncompleted SSL handshakes when in --exec* listening mode make
+ Ncat consume 100% cpu(core/thread).
+ Possible solutions:
+ o Listen on the union of the two sets in ncat_listen.c composed of the
+ current set and a secondary one, ssl_pending which should include the
+ pending ssl hanshake sockets.
+ o Timeout ssl handshakes.
+ o Delay adding the exec output pipes to fselect/WaitForMultipleObjects
+ until the ssl handshake has been completed.
+ http://seclists.org/nmap-dev/2011/q2/988
+---
+
+o Fix ncat.xml(the input for the man page) examples section. - David came up
+ with the final right fix on this one.
+
+o Ncat should close its socket and refuse further connections after the first
+ one, if invoked without --keep-open. That's what traditional netcat does
+ too. - DONE [r24197]
+ http://seclists.org/nmap-dev/2011/q2/944
+ o Add TEST in ncat-test.pl - DONE [r24373]
+
+o Closing Zenmap without stopping the scan first will leave nmap running in
+ the process list on Windows. [r24308]
+ [Actually, Zenmap was unable to kill the nmap scan processes at all on
+ Windows]
+
+o Zenmap should wait for the return exit code of the nmap scanning subprocess
+ upon killing it(canceled scan), otherwise the subprocesses will enter a
+ defunct(zombie) state.[r24235]
+
+o Fix build_icmp_raw and build_igmp_raw filling the packet data payload
+ with zeroes instead of the supplied random data, when nmap is invoked
+ with --data-length.[r24127]
+
+o Investigate and document how easy it is to drop Ncat.exe by itself
+ on other systems and have it work. [r24242]
+ http://seclists.org/nmap-dev/2011/q2/1090
+
+ o We should also look into the dependencies of Nmap and Zenmap.
+ It may be instructive to look at "Portable Firefox"
+ (http://portableapps.com/apps/internet/firefox_portable) which is
+ built using open source technology from portableapps.com, or look at
+ "The Network Toolkit" by Cace
+ (http://www.cacetech.com/products/network_toolkit.html).
+
+o --max-conns is broken in latest svn -> fixed in r24130, other two
+ bugs discovered:
+ o --max-conns 0 kills ncat with a glibc assertion error on calloc with
+ zero as nmemb(??) at:
+ init_fdlist(&broadcast_fdlist, o.conn_limit);
+ o When killing the first initiated connection on --max-conns > 1 Ncat:
+ Ncat: Program bug: fd (5) not on list. QUITTING.
+ [DONE]The previous two bugs were introduced in r24130, they are now fixed
+ in r24193.
+