document security

Lately I need to encrypt a few pdf and zip them securely. Have looked into and enjoyed using pdftk tool in centos and zip tool. Here is the short memo how to use the tool. Remember to test it out yourself first to verify.

=====Practical Example of pdftk from man page ====

       Collate scanned pages
         pdftk A=even.pdf B=odd.pdf shuffle A B output collated.pdf
         or if odd.pdf is in reverse order:
         pdftk A=even.pdf B=odd.pdf shuffle A Bend-1 output collated.pdf

       Decrypt a PDF
         pdftk secured.pdf input_pw foopass output unsecured.pdf

       Encrypt a PDF using 128-bit strength (the default), withhold all permissions (the default)
         pdftk 1.pdf output 1.128.pdf owner_pw foopass

       Same as above, except password ’baz’ must also be used to open output PDF
         pdftk 1.pdf output 1.128.pdf owner_pw foo user_pw baz

       Same as above, except printing is allowed (once the PDF is open)
         pdftk 1.pdf output 1.128.pdf owner_pw foo user_pw baz allow printing

       Join in1.pdf and in2.pdf into a new PDF, out1.pdf
         pdftk in1.pdf in2.pdf cat output out1.pdf
         or (using handles):
         pdftk A=in1.pdf B=in2.pdf cat A B output out1.pdf
         or (using wildcards):
         pdftk *.pdf cat output combined.pdf

       Remove ’page 13’ from in1.pdf to create out1.pdf
         pdftk in.pdf cat 1-12 14-end output out1.pdf
         or:
         pdftk A=in1.pdf cat A1-12 A14-end output out1.pdf

       Apply 40-bit encryption to output, revoking all permissions (the default). Set the owner PW to ’foopass’.
         pdftk 1.pdf 2.pdf cat output 3.pdf encrypt_40bit owner_pw foopass

       Join two files, one of which requires the password ’foopass’. The output is not encrypted.
         pdftk A=secured.pdf 2.pdf input_pw A=foopass cat output 3.pdf

       Uncompress PDF page streams for editing the PDF in a text editor (e.g., vim, emacs)
         pdftk doc.pdf output doc.unc.pdf uncompress

       Repair a PDF’s corrupted XREF table and stream lengths, if possible
         pdftk broken.pdf output fixed.pdf

       Burst a single PDF document into pages and dump its data to doc_data.txt
         pdftk in.pdf burst

       Burst a single PDF document into encrypted pages. Allow low-quality printing
         pdftk in.pdf burst owner_pw foopass allow DegradedPrinting

       Write a report on PDF document metadata and bookmarks to report.txt
         pdftk in.pdf dump_data output report.txt

       Rotate the first PDF page to 90 degrees clockwise
         pdftk in.pdf cat 1E 2-end output out.pdf

       Rotate an entire PDF document to 180 degrees
         pdftk in.pdf cat 1-endS output out.pdf

====================================
How to zip a folder containing file and encrypt the whole folder

example file folder:
myfiles/doc1.txt
myfiles/doc2.txt

cmd:
# zip -r output.zip myfiles -e
(then type in password when prompted)


========

Credit Sources

http://linux.about.com/od/commands/a/blcmdl1_zipx.htm
http://linux.about.com/od/commands/l/blcmdl1_zip.htm
http://linuxpoison.blogspot.com/2010/07/merge-or-encrypt-decrypt-pdf-files.html

experience configure citrix xenserver migration

a few important about configuring citrix xenserver live migration:

set up NFS share resource between xenserver hosts:
- my host running Xenserver 5.6 service pack2. setting up nfs was not easy as service using portmap or dynamic port, making setting firewall iptables rules difficult. => solution: in nfs config file, set static port and set corresponding firewall rules to open those ports.

set up live migration of vm between xenserver hosts:
- storage for vm must not located on any physical xenserver hosts, but should stay on shared NFS share storage resource.
- my xenserver host1 has 4 NIC, setting default vm on host1 has 4 NIC while my xenserver host2 has only 2 NIC. Thus, seeing error "cannot see network" because host2 only have 2 NIC thus any vm on host2 can only have 2 NIC. => insight/solution: delete extra unneeded NIC on vm, left 1-2, then problem solve. vm has same number of NICs, which are supported in either xenserver hosts.
- vm must have xen tools installed - else menu live migration won't exist.

Summary:
- set up NFS is tricky, take long time 4-5 hours but it must be done else nothing work
- some pitfalls in configuring vm for live migrations to watch out ( take another 4-5 hours)

total 1-2 full works day to configure live migration. End of day experience of working live migration is sweet and so well worth it.

developers' book to read

source
http://stackoverflow.com/questions/1711/what-is-the-single-most-influential-book-every-programmer-should-read

• Code Complete (2nd edition) by Steve McConnell
• The Pragmatic Programmer
• Structure and Interpretation of Computer Programs
• The C Programming Language by Kernighan and Ritchie
• Introduction to Algorithms by Cormen, Leiserson, Rivest & Stein
• Design Patterns by the Gang of Four
• Refactoring: Improving the Design of Existing Code
• The Mythical Man Month
• The Art of Computer Programming by Donald Knuth
• Compilers: Principles, Techniques and Tools by Alfred V. Aho, Ravi Sethi and Jeffrey D. Ullman
• Gödel, Escher, Bach by Douglas Hofstadter
• Clean Code: A Handbook of Agile Software Craftsmanship by Robert C. Martin
• Effective C++
• More Effective C++
• CODE by Charles Petzold
• Programming Pearls by Jon Bentley
• Working Effectively with Legacy Code by Michael C. Feathers
• Peopleware by Demarco and Lister
• Coders at Work by Peter Seibel
• Surely You're Joking, Mr. Feynman!
• Effective Java 2nd edition
• Patterns of Enterprise Application Architecture by Martin Fowler
• The Little Schemer
• The Seasoned Schemer
• Why's (Poignant) Guide to Ruby
• The Inmates Are Running The Asylum: Why High Tech Products Drive Us Crazy and How to Restore the Sanity
• The Art of Unix Programming
• Test-Driven Development: By Example by Kent Beck
• Practices of an Agile Developer
• Don't Make Me Think
• Agile Software Development, Principles, Patterns, and Practices by Robert C. Martin
• Domain Driven Designs by Eric Evans
• The Design of Everyday Things by Donald Norman
• Modern C++ Design by Andrei Alexandrescu
• Best Software Writing I by Joel Spolsky
• The Practice of Programming by Kernighan and Pike
• Pragmatic Thinking and Learning: Refactor Your Wetware by Andy Hunt
• Software Estimation: Demystifying the Black Art by Steve McConnel
• The Passionate Programmer (My Job Went To India) by Chad Fowler
• Hackers: Heroes of the Computer Revolution
• Algorithms + Data Structures = Programs
• Writing Solid Code
• JavaScript - The Good Parts
• Getting Real by 37 Signals
• Foundations of Programming by Karl Seguin
• Computer Graphics: Principles and Practice in C (2nd Edition)
• Thinking in Java by Bruce Eckel
• The Elements of Computing Systems
• Refactoring to Patterns by Joshua Kerievsky
• Modern Operating Systems by Andrew S. Tanenbaum
• The Annotated Turing
• Things That Make Us Smart by Donald Norman
• The Timeless Way of Building by Christopher Alexander
• The Deadline: A Novel About Project Management by Tom DeMarco
• The C++ Programming Language (3rd edition) by Stroustrup
• Patterns of Enterprise Application Architecture
• Computer Systems - A Programmer's Perspective
• Agile Principles, Patterns, and Practices in C# by Robert C. Martin
• Growing Object-Oriented Software, Guided by Tests
• Framework Design Guidelines by Brad Abrams
• Object Thinking by Dr. David West
• Advanced Programming in the UNIX Environment by W. Richard Stevens
• Hackers and Painters: Big Ideas from the Computer Age
• The Soul of a New Machine by Tracy Kidder
• CLR via C# by Jeffrey Richter
• The Timeless Way of Building by Christopher Alexander
• Design Patterns in C# by Steve Metsker
• Alice in Wonderland by Lewis Carol
• Zen and the Art of Motorcycle Maintenance by Robert M. Pirsig
• About Face - The Essentials of Interaction Design
• Here Comes Everybody: The Power of Organizing Without Organizations by Clay Shirky
• The Tao of Programming
• Computational Beauty of Nature
• Writing Solid Code by Steve Maguire
• Philip and Alex's Guide to Web Publishing
• Object-Oriented Analysis and Design with Applications by Grady Booch
• Effective Java by Joshua Bloch
• Computability by N. J. Cutland
• Masterminds of Programming
• The Tao Te Ching
• The Productive Programmer
• The Art of Deception by Kevin Mitnick
• The Career Programmer: Guerilla Tactics for an Imperfect World by Christopher Duncan
• Paradigms of Artificial Intelligence Programming: Case studies in Common Lisp
• Masters of Doom
• Pragmatic Unit Testing in C# with NUnit by Andy Hunt and Dave Thomas with Matt Hargett
• How To Solve It by George Polya
• The Alchemist by Paulo Coelho
• Smalltalk-80: The Language and its Implementation
• Writing Secure Code (2nd Edition) by Michael Howard
• Introduction to Functional Programming by Philip Wadler and Richard Bird
• No Bugs! by David Thielen
• Rework by Jason Freid and DHH
• JUnit in Action

how to enable network level authen in windows XP

When windows XP user wants to remote desktop RDP into a more secured windows server such as Windows server 2008, he may be rejected because his windows xp machine does not have network level authentication.

The following short article show how to do that, given he installed SP3 service pack 3. Repost from source credit: https://support.soundenterprises.com/?_m=knowledgebase&_a=viewarticle&kbarticleid=221

===========


When connecting to a Windows 2008 Server using remote desktop from a Windows XP client running service pack 2 or earlier, you get the following error message:

The remote computer requires Network Level Authentication, which your computer does not support.

To enable NLA in XP machines; first install XP SP3, then edit the registry settings on the XP client machine to allow NLA

• Configure Network Level Authentication

1. Click Start, click Run, type regedit, and then press ENTER.
2. In the navigation pane, locate and then click the following registry subkey: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa
3. In the details pane, right-click Security Packages, and then click Modify.
4. In the Value data box, type tspkg. Leave any data that is specific to other SSPs, and then click OK.
5. In the navigation pane, locate and then click the following registry subkey: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders
6. In the details pane, right-click SecurityProviders, and then click Modify.
7. In the Value data box, type credssp.dll. Leave any data that is specific to other SSPs, and then click OK.
8. Exit Registry Editor.
9. Restart the computer.


Source Credit:
https://support.soundenterprises.com/?_m=knowledgebase&_a=viewarticle&kbarticleid=221

remove linux from windows-ubuntu dual boot

ideas

while working intensively on my final project for financial engineering, I start to wonder of new questions as ideas for my future research and study:

- freemind software reminds me when data get large and people at lost, visualization can be extremely valuable. It's good to be able to brainstorm ideas down in a visual structure. It helps to organize ideas especially when working on long paper. So the questions are:
- how to encode and represent knowledge or rules in a knowledge base, ruled based system? format may be text, drawing, video... How to set up a mapping or a user customized meaningful correlation between these representations?

- how to visualize and solve abstract concepts visually?
- how to visually encode knowledge?

I don't have the answers for these questions now...because I'm busy working on this final project paper and have not thought out those questions yet! but I think sometimes it is important to come up and ask the right questions first...to drive direction for answer rather than looking for some random answer blindly...again...another question and ideas about the approach of problem solving...
- is it active questioning/asking driven? or is it "search"/answer driven? how to search if you dont know what to search for????

Install new VM in Centrix XenCenter

Here are a few ways to install/instantiate a new VM in Centrix XenCenter. Some I have learned the hard way.

option1: install from online repositories source
ex: install CentOS. You can find some online repositories. Otherwise, you can follow instruction in User guide for XenCenter to set up your own Redhat repositories.

option2: install from DVD - ISO files
this is the common method we set up a regular machine. Somehow, at first few attempts, I have problems figuring out how to attached DVD or select install from DVD when creating new VM. The KEY is: if you select any existing template in XenCenter when create new VM, you CANNOT select the option to install from ISO library or DVD. Also, it's stupid, but don't bother about attaching DVD to VM because they already have.

the solution is to set TEMPLATE for new VM as "Other install media" in the drop down list of template. Then you will have the options enabled for install from DVD or ISO library. Alternatively, you can leave the Installation Media section empty, then once the VM is created and started the first time, in the "Console" tab dropdown, select the desired DVD or ISO library.

note:
- when create new ISO library such as CFIS/SMB windows share, specify path as //server/share with valid login account
- alternatively, can use a trick to install from URL by running a local Python built in web server serving ISOs files from local with the command:
> python -m SimpleHTTPServer

Life is good again;)

QP

evaluating tools for statistical data mining and analysis of financial application

Many solutions existing including Matlab, C/C++. Java and Python

I choose Python for my projects because:
- scripting language = fast development
- easily translatable/migrate from Java OOP and integration through JPython or with C
- hook up well with R for stat analysis
- beautiful graphing
- free + open source
- great tutorials, documentation

restoring dell laptop running windows vista

My girlfriend laptop Dell XPS M1330 running vista has more likely been corrupted with virus and is mal functioning. A few inportant steps to note for myself that may not be obvious:

- backup important data by manually copying User folders to external hard drive using an ubuntu linux cd
- perform factory backup *** by clicking F8 on boot up=> system repair -> login and choose factory recovery... This step is not obvious and there is no menu option in regular windows log in to do this.

http://support.dell.com/support/topics/global.aspx/support/kcs/document?c=us&cs=RC956904&l=en&s=hied&docid=DSN_336966&isLegacy=true

A week on "network path not found" error

This whole week, I have been reading madly to set up Active Directory and Domain Name service on windows server 2008 and windows 7 clients. Learned that DNS is the KEY to understand and configure AD-DS. But what's more? before even configure DNS, set up computers to connect and share/access resource can be tricky.

The error: network path not found
when I try to join a client computer to a domain. Thought it must be some AD-DS error. But hang on, let me try to do simple file sharing...and yes...I got the same error. The frustration is that I can ping and dns test with nslookup fine the server. Even when all firewalls on server are turned off.

the solution.
Network adapter setting must be set properly for
clients: - check Client for Microsoft Networks
server: File and Printer Sharing for Microsoft Network <= ****** key solution without this no effort works - and I have figure it out today Friday after a long struggling week with nothing done but madly reading.

configure multiple virtual host on apache server on fedora

A few note to myself the necessary steps to configure virtual hosts on apache server running Fedora:

- first need to get DNS mapping correct first. Test ping machines/hosts to verify dns working correctly.
- configuration file = /etc/httpd/conf/httpd.conf
- check that this is the correct configuration file that control the apache server by redirect default web page to a test page and make sure it is displayed correctly. It may be necessary to use different browsers on different computers to access the web server to verify that the server is working properly. Sometimes, I find using the same browser in one machine like firefox seems to cache data and create wrong illusion that the server is always available even when I shutdown the web server.

- modify configuration file to uncomment and enable NameVirtualServer line

- virtual host means using 1 instance of apache server to server multiple web domains/hosts.

- sample configuration for one virtual host:


ServerName www.test.com
DocumentRoot /var/www/


Once >=1 virtual host is configured and used, then every host is considered a virtual host and must have its own configuration as the segment of code above.

- finally restart the web server as below:
#> /sbin/service httpd restart

Done ;)

more on R&D

possible master thesis:

-comparative contemporary programming languages
- high performance programming language
- how to integrate modern programming languages
- concurrent mutation testing

- research work on motivation for programming languages design study as motivated by the need to generalized knowledge in web development

some of my current thoughts regarding R&D directions

Having taken courses in Data Mining, Software Testing, and Parallel computing, I have been equipped with a strong foundation to move on with my research directions. Most of the knowledge I know of now, is still feel theoretical and they demand great applying to in order to solve any real world problems or write a descent thesis.

pending thought for upcoming conquer:
- concurrency programming and testing
- image processing: more application of data mining on image domain, maybe some intro to computer graphic and computer vision
- financial engineering: on simulation modeling, AI agent programming, data mining
will need to find an answer and pick one by end of summer

some R&D directions:
thesis related
- more testing concurrent/parallel/non deterministic system => concurrent programming seems to be the best fit as of now. But thinking about getting more coverage of MS computer science with image processing component and more practical domain application of data mining. On the other hand, image system is static, but financial system is dynamic and non deterministic and exist opportunity for data mining as well....testing maybe a bit hard..and can still have computer graphic in next spring

work related
- computer security - a must
- research on web application development in respect to data centric, content management system

IBM developer work has some great articles today that interest me:

use python to parse unix memory dump
http://www.ibm.com/developerworks/linux/library/l-parse-memory-dumps/

basic use of Pthread
http://www.ibm.com/developerworks/linux/library/l-pthred/index.html
=> remind me of applying my parallel computing knowledge, learn more about cache friendly algorithm

distributed concurrent programming with Erlang(!)
http://www.ibm.com/developerworks/opensource/library/os-erlang2/
=> remind me of all the new interesting language I expose to this semester and to continue my study of programming language and a vote for taking concurrent programming as multi core programming demand and combination of parallel and testing.: python, D, lua....

my upcoming project: build open source content management system
http://www.ibm.com/developerworks/opensource/tutorials/os-cms2/

automate test case with rational test manager
http://www.ibm.com/developerworks/rational/library/automate-test-cases/index.html

how to use Wikipedia API to get its info: - great potential for data mining and content management system
http://www.ibm.com/developerworks/library/x-phpwikipedia/

IBM developer work is cool with a whole sections for Linux, Java, and webdevelopment
http://www.ibm.com/developerworks/web/
=> browsing these IBM technicals inspired and reminds me to make my own collection of technical documents that I have learned and tried

interesting tutorial on learning HOW to think/program functionally
http://www.ibm.com/developerworks/java/library/j-ft1/index.html

some tutorial on personal finance on lifehacker
http://lifehacker.com/5803422/the-graduating-students-guide-to-managing-finances-and-tackling-debt

...
I start to think....since now I have well grounded...should I focus on my research topic?!


interesting tutorial javascript for Java developer
http://www.ibm.com/developerworks/java/library/j-javadev2-18/index.html

even more research direction: multithreaded data structure for parallel computinghttp://www.ibm.com/developerworks/aix/library/au-multithreaded_structures1/
=> remind me that thinking master data mining or high performance computing simply by taking a graduate course is definite not enough...APPLY >>>>APPLY>>>>APPLY

IBM hadoop
http://www.alphaworks.ibm.com/tech/idah


****** IBM Computer Science Research projects
http://domino.research.ibm.com/comm/research.nsf/pages/d.compsci.html

IBM programming language and software engineering research
https://researcher.ibm.com/researcher/view_pic.php?id=3
https://researcher.ibm.com/researcher/view_grouppubs.php?grp=3

=> remind me to make my own website with research projects FOCUS on real practical hand on project that I have done regarding programming languages..+.testing..+.high performance computing = testing high performance language?! another vote again for concurrent programming and testing. Should try to get something as descent as data mining term project going...active research.

top free open source web development tool
http://www.andrewsellick.com/34/top-15-free-and-open-source-web-developer-tools-updated

academic research in programming language design and compiler
http://www.cs.cmu.edu/~mleone/language-people.html

software testing by statistical method
http://www.itl.nist.gov/div897/ctg/stsm.htm

workshop on programming language for HPC
www.cs.uiuc.edu/~wgropp/bib/reports/HPCWPL.pdf

REMIND MYSELF: to INVOLVE in an open source project ASAP, especially project on parallel programming language such as Scala or Fortress for java, OpenJDK
http://en.wikipedia.org/wiki/Fortress_%28programming_language%29
http://openjdk.org/

Focus on my major concentration. Financial engineering can be a side reading...as open video available anyway...why pay? => more vote on taking concurrent programming, focus to improve my weak point, then take CG later

IBM University Relation: resource for students, also entrepreneurs
http://www.ibm.com/developerworks/university/

system security guideline for NSA
http://lifehacker.com/5803569/lock-down-your-computer-like-the-nsa

graphic assembly programming

http://www.gamedev.net/page/resources/_/reference/programming/140/283/graphics-programming-black-book-r1698

http://people.cs.vt.edu/~feng/index.php

http://developer.nvidia.com/cuda-spotlight-compute-cure

Open source video lecture

in continuing example of MIT, IIT India:
http://nptel.iitm.ac.in/courses.php?branch=Comp
http://onionesquereality.wordpress.com/2008/04/30/more-video-lectures-iit-open-course-ware/

mail server on linux

To figure out what mail server running on linux:

alternatives --display mta

alternatives --config mta

sending email:
> sendmail emailaddress

more sendmail tutorial
http://rimuhosting.com/support/settingupemail.jsp?mta=sendmail

open source software alternative

http://www.lostintechnology.com/windows/20-best-free-or-freemium-alternatives-to-popular-paid-software-programs

http://www.lostintechnology.com/windows/5-great-examples-of-cross-platform-open-source-software

sleeping on data mining research

This spring semester, I am working on a data mining research relating to software testing and parallel computing. Some of my current research questions:

- how can data mining help software testing?
- what are the challenge of software testing? as of now, I know that the input space for testing - or test space is infinitive - thus testing will be complicated and expensive - where I think data mining tech come into help by reducing this infinite search space into a cost effective critical small subset of tests needed?
- what is the most minimal test set- test requirement? i think a lot of research and study has come up with quite some metrics of test requirement, coverage for this... the next question is how to evaluate goodness of 1 test set to others? = how to evaluate goodness of 1 cluster of tests with another cluster? by what metrics?

- as of now, I envision:
testing challenge = infinite state input space, infinite and costly tests
desired outcome= a group of minimalist cost effective critical tests
tool = data mining clustering algorithm
project scope: clustering analysis of test space
s1= for imperative program
s2=logic programming/finite state machine
s3 = parallel and concurrent program

most of the reference library books I have read so far all talking about logics, concurrent programming and testing. The question that bothering me is what is the relation between logic and testing and concurrency?????????????????? why what what do I have to deal with logic???

A journey into VirtualBox and Cluster setup

tutorial on how to share data between host and guest
http://blogs.sun.com/tao/entry/virtual_box_shared_folder_between
http://www.dedoimedo.com/computers/virtualbox-guest-addons.html

other vm setup
http://www.virtualbox.org/manual/ch05.html#vdidetails
http://forums.virtualbox.org/viewtopic.php?t=674
http://www.virtualbox.org/manual/ch05.html#vdidetails
http://www.virtualbox.org/manual/ch05.html#cloningvdis

helpful forum
http://forums.virtualbox.org/viewtopic.php?t=674

Rubrics in less than 5 minutes

So recently I have attended the grading workshop for graduate teaching assistant. One of the interesting grading tool that I got introduced to is called RUBRICS. I like it and thought it may be useful for me. So without further due, here is my attempt.

Rubrics in 5 minutes

What is RUBRICS?
- is a grading tool
- help you save time grading tons of paper
- make sure your grading is consistent from begining to last paper
- provide timely and correct feedback to student for better and critical learning

RUBRICS components:
1- assignment title
2- description
3- performance scale: use encouraging words, use 3-5 scales for optimal result
4- evaluation dimension

Example1:
Title=Rubrics for Identify Good Programming Practices Assignment
Description=this assignment is to test that students can identify and
explain good programming practice
performance scale= excellent/competent/need work
evaluation dimension:
- basic knowledge: be able to identify good programming practice elements ( give
a number - maybe 14 elements: comments, constant, variable, scoping)
- apply: can give a realistic example of software using good programming practice
- evaluation: can explain the example, show the pros and cons of the software, the
good elements, great if can show limitation, and suggest improvement or specific
good programming practice to make example software better.


example2: rubrics for presentation
TBD

Reference
- see book