Thursday 28 October 2010

Kernel code and compiling

LKM (loadable kernel module) -> alter without reboot, load/unload when needed, smaller core kernel. Example : device driver

as opposed to SK (static kernel) -> need reboot when altered


xxx.c --> xxx.ko (ko= kernel object)
.so = shared object

A. /proc/kallsyms contains all function of kernel (ex. printk) and the services using it.
===> printk is print at kernel (seen at dmesg or /var/log/messages)
===> printf is print at user mode

B. /boot/System.map.xxxxx contains all the system calls (which is a "call" to kernel space, from user space)

Symbols that are visible for Load kernel module is in:
C. /lib/modules/'uname -r'/build/Module.symvers


ABC relation:
C containing the symbol (function/call) and the memory address (address important for debugging). B containing all the symbol(along with the ID) with the services use it (ex. service : ip, ipv6,nfs). A containing the symbol/function/call(along with the ID) available, and with subs of that function.





To compile:

Make the xxx.c file (C source code)
Cretae the Makefile (enough this 1 line ==> obj-m := xxx.o )

compile it by:
make -C /usr/src/kernels/2.6.18-92.el5-i686/ modules M=$PWD

Result many files, one of them is xxx.ko

modinfo xxx.ko (To see the information which is written in the xxx.c)

insmod xxx.ko (opposed : rmmod xxx.ko or rmmod xxx)
==> must use the path/file location, and execute/install module dependency beforehand
==> modules and its alias is in /etc/modules.conf (either can be used)
==> dependency is in /lib/modules/version/modules.dep

modprobe xxx (opposed : modprobe -r xxx)
==> modprobe use insmod too, but it is aware the def location of module /lib/modules/version, and it is aware of the dep. So use this better.

lsmod (to see what is installed) ==> /proc/modules

depmod -a (to refresh the module database list) IMPORTANT!!

To make it (copy auto) to /lib/modules/.........
make -C /usr/src/kernels/2.6.18-92.el5-i686/ modules_install M=$PWD

Monday 25 October 2010

Memory physical types

RAM vs SAM (random addresses vs serial addresses(tape) )

DRAM vs SRAM
*Dynamic(keep refreshed the voltage, up to 1000times/sec)
*Static (need not refreshed, so faster but more expensive)

Note : memory chips containing transistors (each transistor has capacitor for storing electric as represent of bit 1), the transistor function as switch, 0 or 1. Transistor has Base, C and Emitter. These 3 legs make a transistors as electronic switch.

DRAM vs SDRAM

* Synchronous : faster and more expensive (it has feature pipelines, where it can do next task without wait the prev one). The sync is about time clock, so it has cycle to do actions.

== Pipelining means that the chip can accept a new instruction before it has finished processing the previous one. In a pipelined write, the write command can be immediately followed by another instruction without waiting for the data to be written to the memory array. In a pipelined read, the requested data appears after a fixed number of clock pulses after the read instruction, cycles during which additional instructions can be sent. (This delay is called the latency and is an important parameter to consider when purchasing SDRAM for a computer.) ====

Complete on : http://en.wikipedia.org/wiki/Synchronous_dynamic_random_access_memory


DIMM vs SIMM

* dual inline memory module



DDR (dual data rate)vs SDR (single)
*


DDR2 ( is then DDR1)
DDR3 (is faster in transfer than DDR2)



x4 vs x8 (by 4 vs by 8)

Sunday 24 October 2010

How raid 5 works and the parity calculated

RAID 5, required 3 disks minimum. The total storage afterward is (n-1) disk capacity, and YES all disks must have same size.

How they calculate it the parity is like this:

disk1 disk2 disk3 .... diskLast
0 1 0 parity:disk1 XOR disk2 then the result is XOR to disk3
parity 0 0 1

Above example of disk stripe segment of 1 bit, in real world the normal ones is between 4bits to 256bits.


The magic is when disk1/2/3 failed, it can rebuild from the result, as following:
disk2 XOR disk3 then XOR to parity(above is in diskLast) = disk1

It is not magic though, it is only about:
if bits same then it friend (0 XOR 0 = 0 (friends) 1 XOR 1 = 0 (friends) )
if bits differs then it is enemy.
So from the friend/enemy, you can figure the other bit from the partner bit.

Here is the best explanation:
http://www.scottklarr.com/topic/23/how-raid-5-really-works/

The other thing you must know, when the drive failed, the server will use more IO and CPU, cause when read operation happening it will calculate to result the missing data ( i wanna say that everything is in place as normal, NOT like the idea where if a drive failed then the parities are all calculated and stores the missing bits to the parity bits place as replacing, it is NOT like that)

When a new drive comes, rebuild happens and same exactly as the failed one + the parity from the write operation after failed time.

Wednesday 28 July 2010

AIX : setting old printer in AIX

in Printer:
Go to printer pysically -> "test print" by pushing the button which generates an info page => record the ip and hostname (and queue name if exist)

in AIX:
A. run : vi /etc/hosts -> add the ip and hostname
test ping hostname of the printer


B. smit mkpq
configure remote printer
go to 3rd line options : local printer filtering (dont choose the 1st line -> standard printer)
Put the queuename : infoprint (just an example)
Put the hostname : XXXXXX (printer's ip or hostname listed /etc/hosts)
Put the remote queue : pr1 (this is a common queue name, other printer would be same or maybe lp1)
Choose the printer brand -> IBM -> Others -> Generic -> do not make BOOTP or network start
Note : we choose other as driver becuase this is an old printer


C. Tested by:
lpstat -> Checking if the status is READY, not DOWN.
lp -dtest2 /etc/hosts -> will print a singe line
lp /etc/hosts -> will print whole file
lpr -P queuename /etc/hosts -> will print a file in particular queue name

Friday 9 July 2010

RPM - source vs binary

reference : http://www.rpm.org/max-rpm/ch-rpm-build.html

2 kind of packages : binary and source (checked by cmd : file xxxx.rpm)

If it is a source, after you install using "rpm -ivv xxxx.rpm" will generate:
1. patch(es) file
2. SPEC file --> is similar with a Make file
3. xxx.tgz --> in which some files of .c and .h located

From this point the steps would be:
-> BUILD -> BIN RPM package and SourceRPM package
Note: the first source is the original, this one is the one after build. See below.

>>>>>>>>
The normal directory layout consists of a single top-level directory (The default name is /usr/src/redhat), with five subdirectories. The five subdirectories and their functions are:

*/usr/src/redhat/SOURCES — Contains the original sources, patches, and icon files.
*/usr/src/redhat/SPECS — Contains the spec files used to control the build process.
*/usr/src/redhat/BUILD — The directory in which the sources are unpacked, and the software is built.
*/usr/src/redhat/RPMS — Contains the binary package files created by the build process.
*/usr/src/redhat/SRPMS — Contains the source package files created by the build process.
>>>>>>>>>

Thursday 17 June 2010

List of web builder for e-Commerce FREE

Famous:
http://www.oscommerce.com/
http://www.zen-cart.com/


Well-known (I like this):
http://virtuemart.net/home/demo


Good to have (Not clear either e-commerce or cart-only):
http://sourceforge.net/projects/profbiz-cart/
https://www.storesprite.com/secure/download.php
http://mymarket.sourceforge.net/

Chart Only:
http://www.unibia.com/unibianet/super-simple-shopping-cart-php

Checking http protocol over telnet

This summary is not available. Please click here to view the post.

Tuesday 15 June 2010

C++ pointer

& is reference operator
* is deference operator

<><><><><><><>

int p=10; assuming p address is 1200.
so &p is 1200 and p is 10. In this case : & = address of

<><><><><><><>

POINTER

int * q
q = &p ==> q is containing address!
*q = 11 ==> *q is containing value, in this case : * = value pointed by

Important :
int * q and *q, those 2 * (stars) have diffrent meaning.
One is denotes a pointer (not variable nor constant value).
One is deference operator.

><><><><><><><

End result : p is 11.

Excerpt from : http://www.cplusplus.com/doc/tutorial/pointers/

Monday 14 June 2010

C++ common pitfall

czz61240@LMPH8MF:~/testing123$ g++ -Wall a -o z
/usr/bin/ld:a: file format not recognized; treating as linker script
/usr/bin/ld:a:1: syntax error
collect2: ld returned 1 exit status

>>>>>>>>>>>>>> Because a does not have .c extension, should be a.c


czz61240@LMPH8MF:~/testing123$ gcc a.c -o z
a.c:3:20: error: iostream: No such file or directory
a.c:5: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘namespace’
a.c: In function ‘main’:
a.c:9: error: ‘cout’ undeclared (first use in this function)
a.c:9: error: (Each undeclared identifier is reported only once
a.c:9: error: for each function it appears in.)


>>>>>>>>>>>>>>> Because iostream (successor of iostream.h) is C++ header, so must use g++ compiler.

Friday 11 June 2010

Email : reply-to in headers

IDEA : recipient will reply to specific address, not the sender.

Note: in headers email, X-parameter denotes this parameter is inserted by human.


>>>>>>>>>
To add the Reply-To header to an email, you need to use the MailMessage.Headers property. For example:

mail.Headers.Add( "Reply-To", "alternate_email@mycompany.com")

The following code snippet demonstrates this technique.

[ C# ]

MailMessage mail = new MailMessage();
mail.To = "me@mycompany.com";
mail.From = "you@yourcompany.com";
mail.Subject = "this is a test email.";
mail.Body = "this is my test email body.";
mail.Headers.Add( "Reply-To", "alternate_email@mycompany.com" );
SmtpMail.SmtpServer = "localhost"; //your real server goes here
SmtpMail.Send( mail );


[ VB.NET ]

Dim mail As New MailMessage()
mail.To = "me@mycompany.com"
mail.From = "you@yourcompany.com"
mail.Subject = "this is a test email."
mail.Body = "this is my test email body."
mail.Headers.Add("Reply-To", "alternate_email@mycompany.com")
SmtpMail.SmtpServer = "localhost" 'your real server goes here
SmtpMail.Send(mail)


Reference : http://www.systemwebmail.com/faq/2.7.aspx


LOTUS NOTES:

In new email creation, go to delivery options - advance - reply this memo to.....

Thursday 10 June 2010

Jython to check the Q numbers in JMS

su - wasown -c /applications/WebSphere/AppServer/6.0.0.0/profiles/XXX/bin/wsadmin.sh -lang jython -f /home/wasown/listQueueDepth_WAS60.jy --v


the listQueueDepth_WAS60.jy :

4:root@defrqap100mssxm:/opt/XXX/jms # cat /home/wasown/listQueueDepth_WAS60.jy
import sys
import os
import time
from java.util import *

# global DEBUG setting: 1 = active, 0 = disabled
DEBUG=0

##
## In this sequence, define the partial names of SIB destinations you are interested in!
##
_sibusName = "Saturn_JMS_Bus"
_myDests = ( '_SYSTEM.Exception', 'JMS' )


###########################################################################
###########################################################################
###
### FUNCTIONS
###
###########################################################################
###########################################################################


#get the command line
def _getScriptName():
p = java.lang.System.getProperties();
keys = p.keys();

while( keys.hasMoreElements() ):
name = keys.nextElement()
# print "Property: ", name," - ",
# print java.lang.System.getProperty(name)
return "not implemented in WAS6.0's jython versio!n"


# returns a timestamp string e.g. "2009.01.01 23:47:59"
def getCurrentTimeStamp():
timeArray = time.localtime(time.time())
tstmp = str(timeArray[0]) + '.' + str(timeArray[1]).zfill(2) + '.' + str(timeArray[2]).zfill(2)
tstmp = tstmp + ' ' + str(timeArray[3]).zfill(2) + ':' + str(timeArray[4]).zfill(2) + ':' + str(timeArray[5]).zfill(2)
return tstmp

# returns the name part from any MBean identifier string
# MBean id strings look like:
# "WebSphere:name=JMSTestOutboundQueue,process=server1,platform=dynamicproxy,node=esbNode,SIBus=MediaBus,......"
#
def getNameFromMBeanID(mbeanId):
t1 = mbeanId.split('name=')
t2 = t1[1].split(',')
return t2[0]

def usage(scriptname):
options = "--h --v --d "
print "\n\nusage: ",scriptname, options
print ""
print "\t d secs if is not specified, program exits after one iteration."
print "\t Otherwise it will run forever, sleeping for seconds between iterations."
print "\t v verbose mode for better (human) readability. Non verbose means CSV list output."
print "\t "
print "\n\n"


### MAIN
#
#
#



###############################################################
## getopt start

args = sys.argv
if (DEBUG > 0):
print "len argv = ", len(args)
print "args = ",args
print "-----"



sleepTime = -1
verbose = 0

# process single (toggle) args first
#
argscopy = args[:]

for i in range(0,len(argscopy)):
if (argscopy[i] == "--h"):
print "HELP"
args.pop(i)
usage("")
sys.exit(0)
elif (argscopy[i] == "--v"):
args.pop(i)
verbose = 1

# process parameter args next: e.g. "--d 120"
#
if ((len(args) % 2 ) == 1): args.append(" ")
if (DEBUG > 0):
print "remaining args are:",args

argscopy = args[:]
for i in range(0,len(argscopy),2):
opt = argscopy[i]
val = argscopy[i+1]
if (DEBUG > 0): print "option: ", opt, " - value: ",val
if (opt == "--d"):
sleepTime = val
args.pop(i)
args.pop(i)

if (DEBUG > 0):
print "dbg: cmdline options found:"
print "dbg: sleepTime:",sleepTime
print "dbg: verbose :",verbose
print ""

if (len(args) > 0):
print "unprocessed args: ",args
print

## getopt end
###############################################################


# AdminControl.queryNames() output strings look like this:
#
# WebSphere:name=JMSTestOutboundQueue,process=server1,platform=dynamicproxy,node=esbNode,SIBus=MediaBus,version=6.1.0.23,
# ID=4BF5F7FBBD2C16D3F7D10B5B_QUEUE_10000005,type=SIBQueuePoint,mbeanIdentifier=com.ibm.ws.sib.admin.impl.JsQueuePoint,
# cell=esbCell,spec=1.0,SIBMessagingEngine=esbNode.server1-MediaBus
#

# retrieve all SIBQueuePoints
#
allSIBusDestinations = AdminControl.queryNames("WebSphere:*,type=SIBQueuePoint,SIBus=" + _sibusName).split()

# container for the QueuePoints we are interested in
matchedDests = []


for destMBeanId in allSIBusDestinations:
destName = getNameFromMBeanID(destMBeanId)
if (DEBUG > 0): print "dbg: found destination: ", destName
#
# see if we are interested in this particular SIBus destination
for m in _myDests:
## if (m == destName):
if (destName.startswith(m)):
matchedDests.append(destMBeanId )
# end if
# end for
# end for
if (DEBUG > 0): print ""



# begin CSV output

if (verbose == 0):
# print header row
print "time",",",
for i in matchedDests:
qname = getNameFromMBeanID(i)
print qname,",",


# print data rows
while (1):
print
if (verbose == 0):
print getCurrentTimeStamp(),",",

for d in matchedDests:
print AdminControl.getAttribute(d, "depth"),",",
else:
print getCurrentTimeStamp()
print "-------------------"
for d in matchedDests:
print AdminControl.getAttribute(d, "identifier")+":",
print AdminControl.getAttribute(d, "depth")
## print AdminControl.getAttribute(d, "state")

if (sleepTime == -1):
print ""
sys.exit(0)

time.sleep(int(sleepTime))

Wednesday 9 June 2010

RMI, IIOP, CORBA vs DCOM, RPC

Java Tech Client - Server Process connection

RMI is remote method invocation

CORBA is general tech/architecture, ORB = object request broker (interface to communicate with others P language)

IIOP is CORBA's protocol. CORBA IIOP has its own IDL (I... definition language).

RMI over IIOP is to run RMI to overcome the other P language, so that you dont need to learn the IIOP special language.



Windows Tech

DCOM - Distributed COM : is CORBA in windows

RPC : remote procedure call : the technology connected client and server, there is a spesial dll, called for example : rpcrt4.dll. Some application uses this dll as the interface. This to overcome a developer to create their own dll in their own language.

RPC/IP : used by exchange server and outlook client
RPC/HTTP : used by exchange server and outlook client which has slow connection


Unix Tech
Open Group’s Distributed Computing Environment (DCE) : RPC in UNix


Conclusion:
I would say the DCOM+RPC is equal to CORBA+IIOP.

Reference: http://technet.microsoft.com/en-us/library/cc787851(WS.10).aspx

Unix trick : ctrl - r and ctrl - s

Tips and tricks

ctrl - r ==> find reverse command in history
ctrl - s ==> ctrl - r in forward direction
if ctrl - s does not work : type stty -ixon beforehand.
Keep in mind those command can be pushed over and over again till you find what you want.


alt > ==> repeat cmd from last line in history
alt < ==> repeat cmd from first line in history

In VI:
type r ==> to replace a character after the cursor.
type R ==> to replace some characters after the cursor.

Monday 7 June 2010

process affinity processor

Linux:


xx@XX:~$ chrt --help
chrt (util-linux-ng 2.14.2)
usage: chrt [options] [prio] [pid | cmd [args...]]
manipulate real-time attributes of a process
-b, --batch set policy to SCHED_BATCH
-f, --fifo set policy to SCHED_FIFO
-i, --idle set policy to SCHED_IDLE
-p, --pid operate on existing given pid
-m, --max show min and max valid priorities
-o, --other set policy to SCHED_OTHER
-r, --rr set policy to SCHED_RR (default)
-h, --help display this help
-v, --verbose display status information
-V, --version output version information

You must give a priority if changing policy.
Report bugs and send patches to


xx@XX:~$ taskset -pc 1 32752
pid 32752's current affinity list: 0
pid 32752's new affinity list: 1



xx@XX:$ taskset --help
taskset (util-linux-ng 2.14.2)
usage: taskset [options] [mask | cpu-list] [pid | cmd [args...]]
set or get the affinity of a process

-p, --pid operate on existing given pid
-c, --cpu-list display and specify cpus in list format
-h, --help display this help
-V, --version output version information

The default behavior is to run a new command:
taskset 03 sshd -b 1024
You can retrieve the mask of an existing task:
taskset -p 700
Or set it:
taskset -p 03 700
List format uses a comma-separated list instead of a mask:
taskset -pc 0,3,7-11 700
Ranges in list format can take a stride argument:
e.g. 0-31:2 is equivalent to mask 0x55555555


xx@XX:$ pidstat
Linux 2.6.28-11-generic (XX) 06/07/2010 _i686_ (2 CPU)
01:44:27 PM PID %usr %system %guest %CPU CPU Command
01:44:27 PM 1 0.00 0.00 0.00 0.00 0 init
01:44:27 PM 3 0.00 0.00 0.00 0.00 0 migration/0
01:44:27 PM 4 0.00 0.05 0.00 0.05 0 ksoftirqd/0
01:44:27 PM 6 0.00 0.00 0.00 0.00 1 migration/1

pmap ==> pmap - report memory map of a process

strace ==> strace - trace system calls and signals



WINDOWS:

start cmd_process (you can set the cpu affinity)
taskmgr (you can change the cpu affinity)

Wednesday 26 May 2010

Memory, Virtual memory, /3GB /USERVA, /PAE, PTE

virtual memory --> PTE/page table entry --> memory/paging (RAM/disk)

PTE : is the map table

In windows 32bit: 2GB is for kernel space, the other 2GB for shared/user.
32bit means it can allocate only up to 4GB memory.
The division means that each user process can have up to 2GB of virtual memory address.

/3GB ==> makes each user process can have up to 3GB (3072MB)

/USERVA ==> is a way to offset the lack of memory for PTE (PTE is in kernel space??).
So if /3gb is used, and /userva is used (ex. =3030), the PTE can have additional space as much as the substraction of them (3072-3030MB=42MB additional for kernel space : free PTE).
More detail : http://support.microsoft.com/kb/810371

/PAE ==> is a way to have more than 4GB memory recognised by the OS. It does not mean each process can have more than 2GB (or 3GB with /3GB on), but it means that there are some more space in memory to be used instead to be paged out to disk.


Wanna know deeper about address in memory, go to :
http://www.cs.cornell.edu/~tomf/notes/cps104/virtual.html
http://everything2.com/title/Page+Table+Entry

Tuesday 25 May 2010

Threads in Processes

Threads are in process, they use the same address space of the process.

How to see threads inside process:
Linux: pstree
UNix : ptree

In general:
AIX : ps -mo THREAD -L pid
Linux : ps -L pid

Topas : press H

Manually, just go to : /proc/pid/lwp/threadsID

What can you do:
Linux : renice integerTID (TID = thread ID)
AIX : you can see which processor handle the threads and define which you want, detail:
http://www.ibm.com/developerworks/aix/library/au-aixprocesscontrol/index.html

Can you kill TID in runtime:
Yes, just kill TID, but not suggested, better to kill the process.
In one well-known application, kill a TID same like killing the whole process.
In the programming there is function to stop only thread though, do googling for it.

Monday 24 May 2010

JSP and Servlet

JSP = java server pages (.jsp) = a html-like file which contain dynamic content (eg:date now)

html file extension can be changed into jsp extension, then that file will go through the tomcat translation ==> java file, then it'll be compiled into .class.

JSP basically is a html file with some java code inside (extrainfo: java code in a block is named scriplet, like function in vbs).

This html-jsp can import some java classes.

And can use bean as well, bean basically is a java class (java file compiled --> .class file), bean can have scope session (so can be used at diffrent session at which the bean containing diffrent value@property)
http://www.jsptut.com/Forms.jsp

EJB container manage the lifecycle of Ent bean, for statefull: doesntexist - ready - passive
active is in memory, passive in 2nd storage.
after ejbremove method called by ejbContainer, it is ready to be GC-ed.
http://java.sun.com/j2ee/tutorial/1_3-fcs/doc/EJBConcepts9.html


More detail on this : http://www.jsptut.com/

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Servlet (a java class) : java programming language concentrate on web function

javax.servlet.http.HttpServletRequest
javax.servlet.http.HttpServletResponse
javax.servlet.jsp.JspWriter
javax.servlet.jsp.PageContext

More detail on this : http://www.jsptut.com/Further.jsp

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

So jsp is the presentation, and the servlet is the engine.

Friday 21 May 2010

IBM TDS / LDAP

bash-2.05b# ps -ef | grep ldapdb2
ldapdb2 614418 1106044 0 Mar 24 - 11:28 db2acd 0
ldapdb2 1134664 1106044 0 Mar 24 - 67:01 db2sysc 0

bash-2.05b$ ps -ef | grep ldapdb2svc ==> result 0 (normal - just info)
the process is "db2sysc" which open the 3700 port (this is the ldapdb2svc port)

bash-2.05b# ps -ef | grep ibmdiradm
ldap 638978 1 0 Mar 24 - 0:45 /usr/ldap//bin/ibmdiradm -l

bash-2.05b# ps -ef | grep ibmslapd | grep -v grep
ldap 688256 1 2 Mar 24 - 623:26 /usr/bin/ibmslapd

>>>>>>>>>>>>
Usage: ibmdiradm [options]
where:
options:
-f file use file as configuration file
-h level debug level. Use in conjunction with ldtrc
-p port use port as the port to start ibmdiradm on
-s SSLport use SSLport as the port to start ibmdiradm on
-V print version information
-? this help screen

bash-2.05b# ibmdiradm -V
Directory server version is IBM Tivoli Directory Admin Server(SSL), Version 5.2 (Oct 17 2005 17:13:43)
>>>>>>>>>>

bash-2.05b# ibmslapd --help
ibmslapd: illegal option -- -
Usage:
ibmslapd [ -a | -n | -c ]
where:
-a start server in configuration only mode
-n do not start server in configuration only mode
-c run in console mode
bash-2.05b# ibmslapd -v
Directory server version is IBM Tivoli Directory (SSL), Version 5.2 (Oct 17 2005 17:08:46)

>>>>>>>>>>>>>>>>>

bash-2.05b# ibmdirctl
You must specify a command

Usage: ibmdirctl [options] command -- [ibmslapd options]
where:
command: command to issue to ibmdiradm
must be one of start/stop/restart/status/admstop

start starts the IBM Tivoli Directory Server
stop stops the IBM Tivoli Directory Server
restart stops and starts the IBM Tivoli Directory Server
status displays whether the IBM Tivoli Directory Server is running
statusreturn sets exit code 0=running, 1=starting, 2=stopped
admstop stops the IBM Tivoli Directory Server Administration Daemon


options:
-D adminDn bind DN
-h hostname ibmdiradm hostname
-K keyfile file to use for keys
-N key_name private key name to use in keyfile
-p port ibmdiradm port number
-P key_pw keyfile password
-v run in verbose mode
-w adminPw bind password or '?' for non-echoed prompt
-Y use a secure ldap connection (TLS)
-Z use a secure ldap connection (SSL)
-? this help screen

ibmslapd options are any options the ibmslapd process takes
at startup time. These are ignored if the 'stop' command was issued.

If ibmslapd options are requested, they must be preceded by the '--'.

If the 'admstop' command is issued successfully, the IBM Tivoli Directory
Server Administration Daemon must be restarted manually. (red - ibmdiradm command)

>>>>>>>>>>>>>>>>>>>>>>>>
Example :

Knowing the status of ibmslapd:
ibmdirctl -D cn=root -w xxxxxx status
ibmdirctl -D cn=root -w ? status ==> for password prompt

Note if you get "connection error", try check ibmdiradm process there or not.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
PORTS (/etc/services/):

ldap 389/tcp # Lightweight Directory Access Protocol
ldap 389/udp # Lightweight Directory Access Protocol
ldaps 636/tcp # ldap protocol over TLS/SSL (was sldap)
ldaps 636/udp # ldap protocol over TLS/SSL (was sldap)
www-ldap-gw 1760/tcp # www-ldap-gw
www-ldap-gw 1760/udp # www-ldap-gw
ldap-admin 3407/tcp # LDAP admin server port
ldap-admin 3407/udp # LDAP admin server port
ldapdb2svc 3700/tcp
ldapdb2svci 3701/tcp
DB2_ldapdb2 60000/tcp
DB2_ldapdb2_1 60001/tcp
DB2_ldapdb2_2 60002/tcp
DB2_ldapdb2_END 60003/tcp

ibm-diradm 3538/tcp # IBM Directory Server
ibm-diradm 3538/udp # IBM Directory Server
ibm-diradm-ssl 3539/tcp # IBM Directory Server SSL
ibm-diradm-ssl 3539/udp # IBM Directory Server SSL

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

bash-2.05b$ db2 list node directory

Node Directory

Number of entries in the directory = 2

Node 1 entry:

Node name = IDSINODE
Comment =
Directory entry type = LOCAL
Protocol = LOCAL
Instance name = ldapdb2

Node 2 entry:

Node name = LDAPDB2N
Comment =
Directory entry type = LOCAL
Protocol = TCPIP
Hostname = localhost
Service name = ldapdb2svc

>>>>>>>>>>>>>>>>>>>>>>>>>>

Getting the config of DB2:
bash-2.05b$ db2 get dbmconfig

>>>>>>>>>>>>>>>>>
TROUBLE SHOOTING:

1. check the ldap db2 instance:
bash-2.05b$ db2 connect to ldapdb2 (ensure the set : DB2INSTANCE=ldapdb2)
Database Connection Information
Database server = DB2/AIX64 9.5.5
SQL authorization ID = LDAPDB2
Local database alias = LDAPDB2

2. db2 => LIST DATABASE DIRECTORY

System Database Directory

Number of entries in the directory = 2

Database 1 entry:

Database alias = LDAPDB2B
Database name = LDAPDB2
Node name = LDAPDB2N
Database release level = c.00
Comment =
Directory entry type = Remote
Catalog database partition number = -1
Alternate server hostname =
Alternate server port number =

Database 2 entry:

Database alias = LDAPDB2
Database name = LDAPDB2
Local database directory = /ldapdb2/data
Database release level = c.00
Comment =
Directory entry type = Indirect
Catalog database partition number = 0
Alternate server hostname =
Alternate server port number =

3. ldapsearch -p 389 -b "" -s base objectclass=*

namingcontexts=CN=SCHEMA
namingcontexts=CN=LOCALHOST
namingcontexts=CN=PWDPOLICY
namingcontexts=CN=IBMPOLICIES
namingcontexts=SECAUTHORITY=DEFAULT
namingcontexts=DC=AMM,DC=XX,DC=NL
subschemasubentry=cn=schema
supportedextension=1.3.18.0.2.12.1
.......
supportedextension=1.3.18.0.2.12.37
supportedcontrol=2.16.840.1.113730.3.4.2
......
supportedcontrol=1.3.18.0.2.10.18
security=none
port=389
supportedsaslmechanisms=CRAM-MD5
supportedsaslmechanisms=DIGEST-MD5
supportedldapversion=2
supportedldapversion=3
ibmdirectoryversion=5.2
ibm-ldapservicename=deehqws011oxixm
ibm-serverId=8d6c19c0-30ea-102a-9241-8d6a2b010899
ibm-supportedacimechanisms=1.3.18.0.2.26.3
ibm-supportedacimechanisms=1.3.18.0.2.26.4
ibm-supportedacimechanisms=1.3.18.0.2.26.2
vendorname=International Business Machines (IBM)
vendorversion=5.2
ibm-sslciphers=N/A
ibm-slapdisconfigurationmode=FALSE
ibm-slapdSizeLimit=500
ibm-slapdTimeLimit=900
ibm-slapdDerefAliases=always
ibm-supportedAuditVersion=2
ibm-sasldigestrealmname=xxxxxxxxx----(red:hostname)

4. ldapsearch -b dc=amm,dc=xx,dc=nl '(uid=at*)'
ldapsearch -b dc=amm,dc=xx,dc=nl '(uid=at*)' dn

???????????????????????????????????????????????????????????????

Unix Tip : Process -> ports and viceversa

the basic is :
lsof | grep ldap
lsof | grep firefox

advance:
lsof -i :389
lsof -p pidnumber

>>>>>>>>>>
In linux:
netstat -putanuw

Thursday 20 May 2010

Socket

Sock Application (e.g., Netscape) - OSI layer 7 Application
|
SOCK DLL/library - socket() - binding to a port to make it listen
|
TCP/UDP - OSI layer 4 Transport
|
IP/IPX
|
Modem or Network card
|
Network

Above is also called TCP/IP stack (stack is LIFO - last in first out)

TCP/IP stack is loaded into a memory space.

KERNEL = TCP/IP + NLB driver + network adapter driver

In unix :
driver/module in .so = shared object or .ko = kernel object (related with memory segmentation)
insmod, lsmod, rmmod

Unix Trip : how to get your gateway address

$ arp
Address HWtype HWaddress Flags Mask Iface
d6e-cd-br-cen ether 00:06:d6: C eth0
gate-bro- ether 00:00:0c:07: C eth0

arp -a
d6e-cd-b.com (9.158.) at 00:06:d6:: [ether] on eth0
gate-br.com (9.158.1) at 00:00:0c::: [ether] on eth0

Monday 17 May 2010

WAS : express, base, ND

Express is like tomcat, only WAR no EJB module. Furthermore single instance for dev-prod to simplify the deployment.

Base, able to handle EJB module. Not support cluster and only support server1 Appserver.
Server1 cant be started or stopped using ISC.

ND, support cluter and multi app servers.

Note :
About ISC :
there is a isclite.ear module in config/cells/node/applications/ if ISC is installed.
In base, you may not install ISC, nor defaultapp (snoop/hello), nor sampleApp(plant)
But you are auto installing the ivtApp.ear and query.ear.
IvtApp.ear is used during the firstStep to verifiy instalation result, like start hem and test the ports.

About trial period:
can be tricked by changing the date of the OS.

Friday 14 May 2010

Security 3 : websphere, plugin, IHS key database

IHS --> SSLengine ON
Certificate location
keyfile location
chainfile location

Plugin : CMS type=> .kdb
ikeyman tools => generate self-signed cert
extract it to a .arm file
transfer it to a WAS servers to be put it in key (or trust file)
more precisely is "ADD" in signer certificate (not personal certificate)

WAS : jks = javakeystore => a db file of certicates collection (new generated by ikeyman tool)
a. one.jks => private key and trusted certs are here
b. key.jks and trust.jks => private key in key.jks while trusted certs in trust.jks

ikeyman tools ==> generate self-signed cert in personal certs
extract it to a .arm file
transfer it to a Plugin servers to be put it in key.kdb
==> signer certificates => ADD
I wonder why, we need "add" to signer certs as well in WAS side.

If you use LDAPS in WAS, then you "add" also the ldap server certs in that jks file.

Note : 1. dont forget to set the path in ISC to the correct location of jks file
2. ikeyman tools is GUI, gsk7cmd is CLI,
3. keytool can be used as well to read the jks wihtout password
4. during the creation of jks, it prompts password, this password is encrypted in a stash file.
5. .arm = .cer
6. keyring = keystore
7. signer certificate = trust certs = ssl from client side
8. key = private key = ssl from server side

reference : http://publib.boulder.ibm.com/infocenter/wasinfo/v4r0/index.jsp?topic=/com.ibm.websphere.v4.doc/wass_content/06061801a07.html

Security 2 : certificates, CA, SSL

How browser and SSL web server connected:

1. browsers usually have the "trust root CA" already which means have those public key
2. first it will work in asymmetric method
3. Certificate of web server containing CN/domain name, digital signature CA (result of encrypting priv key CA : maybe with HASH), HASH (or fingerprint), public key of the web server.
4. Pub key in browser will decrypt the dig signature CA
5. It will get the HASH and compare with the HASH in the certificate
This verify if the CA is really the CA, example: this Cert really come form verisign.
6. After browser verify it, it use the public key web server in the certs to encrypt the req along with sessionID
7. Web server got it and decrypt using it priv key.
8. At this point, web server will do the symmetric key encryption using the sessionID as the shared code.

Note : Dont be confused, there are CA key pair and also web server key pair.
Asymmetric encrypt is slower than symmetric.

>>>>>>>>>>>>>>>>>>>>>>>>>>>

In unix use openssl command
First you generate the priv key RSA.
then from that you create a .csr file. CSR : certificate request
which need some input as domain name, etc.....
then you send the .csr to a CA
if it is commercial CZ, they may contact you for verify and to be paid
then CA will send you the .cert files/.arm files
set the cert localtion on the virtual host section of your http
also set the key file location in the same section
also set the SSLCertificateChainFile in the same section.
==> this last file is a plain text file containing the structure intermediate CA up to root CA
==> explain later on in next


>>>>>>>>>>>>>>>>>>>>>>>>>>

CA :
A. private CA
B. commercial CA

1. root CA : the root, issued to and issued by itself, and signed by it self.
2. intermediate CA : the derivatives up to several level, issued by : rootCA/above intermediate CA
issued to : itself. signed by the ROOT CA or above intermCA.

In secure case : root CA is std alone, intermediate CA is in network. You create csr in intermCA, bring it over usb to root CA. you generate the cert, and bring over usb again to intermCA, and install it. Now you have rootCA signed cert at your intermCA.
Now you can create any new certs to be used by web servers.
Remember in web servers, you need set the chain file (in order client browser can auth the intermCA upto the rootCA).

reference: http://usefulfor.com/nothing/2008/03/20/howto-create-an-intermediate-certifica-authority-ca-using-openssl/

chain file looks like:

--BEGIN--
FDARGARGAGAGFDGFDGDCCCC
the intermCA2
DSFSDGDSFHSHSHSD%^GG#TG
--END--
--BEGIN--
the intermCA1
--END--
--BEGIN--
the rootCA
--END--


>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

summary : certificate is public key with some info of your data
can be digitally signed by a CA or commercial CA
digitally sign means as written in digital signature workflow

Security 1 : Asymmetric, symmetric, hash, digital signature, email encryption

Symmetric : shared code
Asymmetric: pair key - public & private

Hash : SHA or MD5 ==> algorithm to make a data into short form, use to verify if data altered or not along the way. Example : email + hash = 010101100 (this is called fingerprint)

Digital signature ==> fingerprint + private key encryption

you sent : email + digital signature (for now we ignore the email encryption)

recipient has your pub key!
recipient also has the hash!
==> first it will hash the email and it will get the fingerprint 010101100
==> second it will decrypt the dig signature with the pub key and it will get the fingerprint
==> A. comparing the fingerprint step 1 and 2 above, if not equal then email was altered!!
B. also if the pub key failed to decrypt at step 2, then it means the private key wrong, which could means it was not sent by the real person (bcoz private key is only had by the real)

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Now, about email encryption:
the encryption is done by the pub key, and is called RSA method.

Sender has the pub key of recipient and encrypt the email with it.
Recipient will use his priv key to decrypt it.

Note : Even if someone else has recipient pub key, it does not mean he can reverse back the data. Only one who has private key can decrypt it.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Friday 16 April 2010

MQ v7 example to make connection

Structure:
Qremote -- transmissionQ(XmitQ) -- channel -- RemoteServer:listener -- RemoteServer:Qlocal

Qremote:
===> Just a shadow of remoteQL
runmqsc
def qr(nameQR ex. QRMTYY) rname( nameQL on remote server : QL.AY) rqmname( nameQM remote : QMCYY) +
xmitq(QMCYY)

Note : xmitq is optional, if u use default,by default it will go to xmitq whose name same as rqmname. That is why if no xmitq with same name with rqmname: 'amqsput QremoteName' will show error.
If you use uncommon name for the xmitQ (different name with the rqmname), you must put parameter xmitq there.


XmitQ
==> Queue Gateway of all Qremote using this value XmitQ as parameter
runmqsc
def ql(QMCYY) usage (xmitq)


Channel
==> Defining to where, hostname and port of remote server.
runmqsc
def chl(qmcxx.qmcyy) chltype(sdr) trptype(tcp) conname ('ip(port)') xmitq(qmcYY)

Note : this channel will be used by this xmitQ

>>>>>>>>>>>>>>>>>>>>>
BTW, you need to set deadletterQ as well:
runmqsc
def ql(DQL)
alter qmgr deadq(dlq)
>>>>>>>>>>>>>>>>>>>>>>


AT REMOTE SERVER:

listener
runmqlsr -t TCP -p 90XX -m qmgrname

Note : after this you create channel receiver : def chl(xxx) chltype(rcvr) trptype(tcp)
dont forget to create qlocal (QL.AY)


BACK TO LOCAL SERVER


TESTING:
runmqsc
ping chl (qmcxx.qmcyy) ==> this one only test if remote listener port open
runmqchl -c qmcxx.qmcyy -m qmgrname ==> activate channel (has dependancy on listener port AND correct name of xmitQ existence)


PUTTING MESSAGE:
amqsput Qremote (qmgrname)
==> remember this is just shadow, message will go to xmitQ name related
-=> So check it with amqsbcg/gbr xmitQname (to see if message stuck in local server)
amqsgbr is more concise than amqsbcg.


CHECKING MESSAGE @remote server:
amqsgbr QL.AY (qmgrname)
amqsbcg QL.AY (qmgrname)

NOW, if you wanna set message goes to DeadletterQ (when everything is ok):
amqsput XmitQname
==> because channel doesnt know where to transfer it (unlike QR which has Qlocal of remote server)
==> And of course the DeadletterQ is in the local server.

NOW, if channel failed, and you use that : amqsput XmitQname,
it will go to XmitQ, test by : amqsgbr xmitQname.


ADDITOINALLY:
crtmqm -q qmgrname (make it as default qmgr)

MQ v7 Triggering amqsreq.exe and amqsech.exe

Those files should be in:
D:\Program Files\IBM\WebSphere MQ\tools\c\samples\bin

That "c" dir is created only if you install the "development toolkit" during the instalation,
or you can run it again the setup.exe of MQ, to add this feature.

NOTE: IF U SEARCH FOR AMQSBCG.EXE, IT IS ALSO THERE AFTER THIS FEATURE INSTALLED

After this, it will be put in PATH variable, so you must close the "cmd" windows and open new one "cmd" in order to get the refreshed "PATH" variable.

"cmd" == start - run - cmd

Thursday 15 April 2010

WAS + proxy (edge server)

Excerpt from http://www-10.lotus.com/ldd/lcwiki.nsf/dx/configuring-a-reverse-proxy-server2


Configuring a reverse proxy server (2.0)

Overview
A reverse proxy server intercepts client requests to Lotus Connections servers and, when possible, returns cached data rather than contacting the true destination server. A reverse proxy reduces the amount of traffic that a back-end server must perform to satisfy duplicate requests for data, while at the same time improving the response time for those requests.
You can set up a reverse proxy by using WebSphere Application Server Edge components (formerly Edge Server). Edge components can control client access to Web servers and improve access to Web-based content over the Internet or a corporate intranet.

Infrastructure
The topology in this deployment uses a cluster of Lotus Connections servers connected to a remote HTTP server, a remote database server, and a remote LDAP server. These are deployed in a private network behind a firewall. Client request are intercepted and handled by the transparent proxy server. Users are not aware that a proxy exists in the configuration.

This deployment was tested on a Microsoft Windows Server 2003 platform. The following figure illustrates the topology used in this deployment:

Specifications
The following table lists the specifications of the systems used in this deployment:
Component Software OS System
Reverse Proxy IBM WebSphere Application Server Edge Components 6.1 Windows Pentium IV CPU, 3.0 GHz, 500 MB RAM
Web Server IBM HTTP Server 6.1.13 with SSL enabled Windows
Linux 2 Dual-Core CPUs, 2.0 GHz, 8 GB RAM
1 Quad-Core CPU, 1.6 GHz, 4 GB RAM
Lotus Connections 1.0.2
Lotus Connections 2.0
IBM WebSphere Application Server 6.1.0.13
IBM WebSphere Application Server 6.1.0.13 Windows
Linux 2 Dual-Core CPUs, 2.0 GHz, 8 GB RAM
Database DB2 9.1.4
Oracle 10g 10.2.0.3 Windows
Linux 2 Dual-Core CPUs, 2.0 GHz, 8 GB RAM

















Step-by-Step instructions
Before deploying this infrastructure, the product and all related components must be installed.

Installing WebSphere Edge Components

1. Download Edge Components (part number: C88UQML): https://w3-104.ibm.com/software/xl/download/ticket.do?openform
2. Launch the Edge Components launchpad.bat file and install the Caching Proxy component. For more information about installing, see the Edge components information center.
3. Start the proxy service: Click Start > Control Panel > Administrative Tools > Services, and then select IBM Caching Proxy and start the service.
4. Enter the URL of the target Web server: Click Start > All Programs > Edge Components > Caching Proxy > Configuration Wizard. Enter the URL of the Target Web Server, and click Next.
5. Enter a User Name and Password for a new administrator account, and click Next. Click Finish and OK.
6. Open the ibmproxy.conf file (usually located in the \cp\etc\en_US directory) and add the following line to the Start new mapping rules section:

Add Pass /pub/* \cp\server_root\pub\en_US\*


Save the file.
7. Configure the proxy: Open Proxy Configuration > Proxy Settings, select the HTTP check box, and click Submit.
8. Click Proxy Performance, deselect the Run as a pure proxy check box, and click Submit.
9. Restart the proxy server and test the configuration: Open a web browser and navigate to "http://Your_Proxy_Server.com/", where is a Lotus Connections feature (such as Activities or Blogs). If the web page for the selected feature appears, the reverse proxy is working correctly.
You can check the reverse proxy logs under: C:\Program Files\IBM\edge\cp\server_root\logs.


Enabling SSL

A. Create a personal certificate on the proxy server

1. Create a key database for proxy server: Click Start > All Programs > IBM WebSphere > Edge Components > Caching Proxy > Start Key Management Utility.
2. Click Key Database File and select New.
3. Enter the following information and click OK:
Key database type: CMS
File Name: ProxyKey.kdb
Location: C:\ProxyDB
4. At the password prompt, enter your password and select the Stash the password to a file? check box.
5. Create a personal certificate: Click Create > New Self-Signed Certificate....
6. Enter the following information and click OK:
KeyLabel: ProxyCert
Version : X509V3
Key Size : 1024
Common name : Your_Proxy_Server.com
Organization : ibm


B. Import a signer certificate from the Web server

1. Extract the PluginCert.arm file from the IBM HTTP Server. (For more information about IHS, see Edge components information center): Open the plugin-key.kdb file (under C:\IBM\HTTPServer\Plugins\etc\) and click OK. Enter your password when prompted, and click OK.
2. Click WebSphere Plugin Key > Extract Certificate... and enter the following data:
Data type: Base64-encoded ASCII data
Certificate file name: PluginCert.arm
Location: C:\
Click OK.
3. Import the IHS Signer Certificate to the proxy server: Click Start > All Programs > IBM WebSphere > Edge Components > Caching Proxy > Start Key Management Utility, and open the ProxyKey.kdb file.
4. Click Signer Certificates > Add, and enter the following information:
Data type : Base64-encoded ASCII data
Certificate file name : PluginCert.arm
Location : C:\
Click OK
5. When prompted to enter a label, type PluginCert, and click OK.


C. Configure SSL on the proxy server

1. In the proxy server's admin console, open Proxy Configuration > SSL Settings.
2. Select the Enable SSL check box.
3. Select the Attempt to cache content on a secure request check box.
4. Enter the kdb location, for example, c:\ProxyKDB\ProxyKey.kdb, in the Key Ring Database file field.
5. Enter the password file location, for example, c:\ProxyKDB\ProxyKey.sth, in the Key Ring Database Password File field.
6. Click Submit.


D. Modify the ibmproxy.conf file
1. Add the following lines to enable Proxy and ReversePass directives:

Proxy /*
http://venturacn24.cn.ibm.com/* :80


Proxy /*
https://venturacn24.cn.ibm.com/* :443


ReversePass http://Your_HTTP_Server.com/*
http://Your_Proxy_Server.com/*


ReversePass https://Your_HTTP_Server.com/*
https://Your_Proxy_Server.com/*


2. Add the following line to enable the SendRevProxyName directive:

SendRevProxyName yes


3. Add the following lines to enable cache directives (optional):
CacheTimeMargin 0 seconds
CacheQueries Always http://Your_HTTP_Server.com/*
CacheQueries Always https://Your_HTTP_Server.com/*
4. Save the ibmproxy.conf file.
5. Restart the proxy server and test the configuration: Open a web browser and navigate to "https://Your_Proxy_Server.com/", where is a Lotus Connections feature (such as Activities or Blogs). If the web page for the selected feature appears, the reverse proxy in SSL mode is working correctly.
You can check the reverse proxy logs under: C:\Program Files\IBM\edge\cp\server_root\logs.

E. Test the configuration
1. Restart the proxy server.
2. Open a web browser and navigate to "https://Your_Proxy_Server.com/", where is a Lotus Connections feature (such as Activities or Blogs). If the web page for the selected feature appears, the reverse proxy in SSL mode is working correctly.
You can check the reverse proxy logs under: C:\Program Files\IBM\edge\cp\server_root\logs.

Recommended reading

* WebSphere Application Server Concepts, Planning, and Installation for Edge Components: http://publib.boulder.ibm.com/infocenter/wasinfo/v6r1/index.jsp?topic=/com.ibm.websphere.edge.doc/welcome.html
* Reverse proxy definition: http://publib.boulder.ibm.com/infocenter/iseries/v5r3/index.jsp?topic=/rzaie/rzaieproxytypes.htm
* WebSphere Application Server Network Deployment: http://www-306.ibm.com/software/webservers/appserv/was/network/edge.html

Ant and Make

Both are "builder" of application.

In instance, Make is used by perl, C, ...

To run it : make Makefile
The makefile contain : the target:dependency:script to run

example:

in C, there are file.C and headerfile.h. file.c is source code. headerfile.h is like library in which there are main function,variable that can be used among the file.c (ex. include stdio.h)

in C, you compile and then linking :
gcc -c thesourcecode then gcc -o theresultprevcommandANDsomelib

in C, when you have (at same folder):
file1.c and header1.h
file2.c and header2.h

to compile : gcc -Wall -ansi -o theprogram code1.c code2.c
Note: see @ bottom page, if you are confused why no need out headerfile.h at above command!!


You'll compile it like this if you dont want to compile all-in-once, example if you just modify one of the file, you dont need to compile all files :

gcc -c file1.c ==> resulting file1.o = a machine code file
gcc -c file2.c
gcc -o theprogram file1.o file2.o

Here are example makefile content:

''''''''''''''''''''''''''''
default: theprogram

code1.o: code1.c header1.h
gcc -Wall -ansi -c code1.c

code2.o: code2.c header2.h
gcc -Wall -ansi -c code2.c

prog: code1.o code2.o
gcc -Wall -ansi -o theprogram code1.o code2.o

clean:
rm code1.o code2.o

cleanall: clean
rm prog
'''''''''''''''''''''''''''''''

target : dependency
script to run

Note: if too long in a row use "\" as splitter.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

step beginning : make makefile
will execute the make default, which dependens on theprogram,
so theprogram is run, but dependens on code1.o and code2.o,
so the code1.o and code2.o will be run, the script to get the codeX.c, the dependency.


step final: make clean
to remove the machine code files.


>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

in perl, as i remember:

make ==> to prepare the environment
make test ==> to test if any dependency failed
make makefile ==> run the instalation
make clean ==> cleanup some junk during instalation

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Ant, is java-based tool, can be use in vary programming, including replacing Make,
it is more advance with more feature.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Excerpt from http://www.unix.com/programming/47881-help-compiling-header-files-c-file-using-gcc.html

There are two basic ways to compile header files in C:

AAA

If the headers are in the same location as the source you include them like this:
Code:
#include "myheader.h"

if they are standard headers then inlude like this:
Code:
#include


then compile with
Code:
gcc -o myoutputfile myoutputfile.c

BBB

If the headers are located in some other (non-standard) location you would include like standard headers:

Code:

#include
and then use the -I (uppercase i ) flag to include the directory containing them.

Code:
gcc -I/path/to/directory/containing/header_name -o myoutputfile myoutputfile.c

or leave them quoted and use on of:
Code:
gcc -iquote /path/to/directory/containing/header_name -o myoutputfile myoutputfile.c

or
Code:
gcc -include /path/to/directory/containing/header_name/header_name.h -o myoutputfile myoutputfile.c

Tuesday 13 April 2010

IHS + WAS + Plugin

Install IHS
==> Put server name as the identification of the "web server name" which will be used for "folder name" of plugin at IHS and WAS, and also log name as unique identity.
==> Put username and password, as it will be used for registering to Http admin.
==> Http admin is a service in windows, but it isnt accessible from browser.
==> Http server is another service created by the installer.

As i remember we dont need to install plugin, since it is along with the IHS installer,
and during the ihs instalation it asked where the WAS directory is.

But incase plugin wasnt there, just install it reside in IHS installer folder called plugins.


On WAS admin console:
==> go to servers = web servers = new
==> Put servername as same as the "web server name" above, as it is a folder name in the ihs plugin config dir.
==> Put hostname as the hostname/DNS name of where the IHS server is.
==> Put the "service name" of IHS correctly as in services.msc (in oder to be able restart webserver using was admin console)


At this point, the plugin is:
1. in IHS is general plugin of WAS default instalation (context root : snoop, hello, etc)
D:\Program Files\IBM\HTTPServer\Plugins\config\webservername

2. in WAS, plugin not available yet.

At WAS console, u must generate plugin (click the webserver and button will be available):
It will be in D:\Program Files\IBM\WebSphere\AppServer\profiles\AppSrv01\config\cells\win2003Node01Cell\nodes\win2003Node01\servers\"webservername you insert when adding webserver"\

* That folder was created when you add new web server, plugin file will be created under it
* This plugin is only for default_host VH and others if any, but not for the admin_host VH
* admin_host VH is VirtualHost where /ibm/console is (WAS admin console login)
* default_host VH is VirtualHost where /snoop, /hello, /UTC (in my case) are.
* notice there is server.xml there, there you can see the webserver username and decoded password.
* you can decode the WAS user's password by using www.sysman.nl/wasdecoder


At WAS console, u must propagate plugin:
WAS will copy above plugin to IHS plugin - webserver name folder.

Restart the IBM Http service.

Now, u have successfully made it.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

For redirecting/loadbalancing the WAS admin console using plugin, i think you must edit it manually the plugin-cfg.xml.

Allowing Web servers to access the administrative console
http://publib.boulder.ibm.com/infocenter/wasinfo/v6r1/index.jsp?topic=/com.ibm.websphere.nd.doc/info/ae/ae/tins_configACAccess.html

Procedure

1. Use the administrative console to change the admin_host virtual host group to include the Web server port (80 by default).
1. Click Environment > Virtual Host > admin_host > Host Aliases > New.

The default port that displays is 80, unless you specify a different port during profile creation.
2. Specify the IP address, or the name of the machine that is hosting the HTTP server.

For example, if you installed a WebSphere Application Server product on a machine that is named waslwaj.rtp.ibm.com, specify the name in this field.
2. Click Apply > Save.
3. Stop and restart the application server.

For example, to access the administrative console of a stand-alone application server, stop and restart the server1 process.
To stop server1, open a command window and navigate to the profile_root /bin directory. Then issue the following command:

./stopServer.sh server1

After receiving the following message, you can restart the application server:

Server server1 stop completed.

To start the application server, issue the following command:

./startServer.sh server1

When you receive a message that is similar to the following message, the server1 process is running:

Server server1 open for e-business; process id is 1719

4. Stop and restart a deployment manager.

For example, to access the administrative console of a deployment manager, stop and restart the deployment manager.
To stop the deployment manager, open a command window and navigate to the profile_root /bin directory. Then issue this command:

./stopManager.sh

Then issue the following command to stop the deployment manager:

./stopManager.sh

After receiving the following message, you can restart the deployment manager:

Server dmgr stop completed.

To start the deployment manager, issue the following command:

./startManager.sh

When you receive a message that is similar to the following message, the deployment manager is running:

Server dmgr open for e-business; process id is 1720

5. Edit the plugin-cfg.xml file to include the following entries:






...
...
...









...
...
...

AffinityURLIdentifier="jsessionid" Name="/ibm/console/*"/>

UriGroup="admin_host_server1_SERVER1HOSTserver1_Cluster_URIs" VirtualHostGroup="admin_host"/>


If your HTTP server has an HTTP port other than 80, add an entry to the VirtualHostGroup:



The port variable is your HTTP server port.

Friday 9 April 2010

MQ 7 on windows

After installer launched, the prequisite ask about the Websphere Eclipse Platform 3.3,
you must install it first, by go to the prereqs folder (in the same installer CD as MQ installer),
then run the IES (Ibm eclipse SDK).

Configure:
1. crtmqm qmanager
2. runmqsc qmanager
3. runmqsc qmanager
4. define qlocal(qname)


To test it:
use the amqsput.exe in the bin directory :
amqsput qlocal qmanager (CATCH CATCH qlocal is at caps lock, if you dint put '' when created it)
type some text
enter twice to end it

Use the amqsget command.


>>>>>>>>>>>>>>>>>>>
Now, difference between amqsput and amqsputc, when u use amqsputc u must set this variable first:

set MQSERVER=SYSTEM.DEF.SVRCONN/TCP/127.0.0.1(1414)
where first parameter is channel with type of SVRCONN (other type can result error)

amqsputc qname qmanager
>>>>>>>>>>>>>>>>>>>>

Wednesday 7 April 2010

WAS and MQ

You can test using UTC.ear (reside in deploy tool of WAS instlation folder)

CCSID : usually 850 (this is ID of the qmanager that u will use) You can see at the properties of the qmanager.
This must be put exactly same in the QCF and Q creation at WAS console.

At WAS - servers - you can ignore the Websphere MQ Server (No need to register to here, this is only for zOS mq).


>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
HERE IS the PROCEDURE

Excerpt from:
http://activexpert.blog.co.in/2009/09/14/integrate-websphere-application-server-and-mq/

WebSphere Application Server can interoperate with WebSphere MQ in the following ways:

* By configuring WebSphere MQ as an external JMS provider using the WebSphere MQ messaging provider.
* By integrating a service integration bus into a WebSphere MQ network of queue managers using the default messaging provider and WebSphere MQ links.
* By integrating WebSphere MQ for z/OS® queues into a bus using the default messaging provider and a WebSphere MQ server.
WebSphere MQ links provide a traditional WebSphere MQ-style solution to connecting resources. WebSphere MQ servers add the ability to directly access WebSphere MQ queues from within a bus.

Method1: No bus: The WebSphere MQ messaging provider
The WebSphere MQ messaging provider does not use service integration. It provides JMS messaging access to WebSphere MQ directly from WebSphere Application Server.

Method2: Multi-bus: WebSphere MQ network as a foreign bus (using WebSphere MQ links)
A WebSphere MQ link provides an indirect connection between a service integration bus and a queue manager within a WebSphere MQ messaging network. With this type of connection, the messaging bus is seen by the WebSphere MQ network as a virtual queue manager, and the WebSphere MQ network is seen by service integration as a messaging bus. A Websphere MQ link allows WebSphere Application Server applications to send point-to-point messages to WebSphere MQ queues (defined as destinations in the service integration bus), and allows WebSphere MQ applications to send point-to-point messages to destinations in the service integration bus (defined as remote queues in WebSphere MQ). The link also allows WebSphere Application Server applications to subscribe to messages published by WebSphere MQ applications, and WebSphere MQ applications to subscribe to messages published by WebSphere Application Server applications. The link converts messages between the formats used by WebSphere Application Server and those used by WebSphere MQ, and handles data conversion of messages.

Method3: Single-bus: Queue manager as a bus member (a WebSphere MQ server)
A WebSphere MQ server provides a direct connection between service integration messaging engines in WebSphere Application Server and queue managers or queue sharing groups in WebSphere MQ for z/OS. WebSphere MQ server is designed to exploit the high availability, and optimum load balancing characteristics provided by a WebSphere MQ for z/OS network. WebSphere MQ server defines the connection and quality of service properties used for the connection, and also ensures that messages are converted between the formats used by WebSphere Application Server, and those used by WebSphere MQ.

=> Installation instructions for setting up a new WebSphere MQ installation to interoperate with WebSphere Application Server.

1. Install a supported version of WebSphere MQ
2. Follow the WebSphere MQ instructions for verifying your installation setup.
3. Configure WebSphere Application Server and WebSphere MQ to interoperate effectively.
4. At Node scope (not Server scope) set the WebSphere Application Server MQ_INSTALL_ROOT environment variable to the directory where WebSphere MQ is installed. WebSphere Application Server uses the MQ_INSTALL_ROOT setting to find the WebSphere MQ libraries for the WebSphere MQ messaging provider. If you use the default messaging provider and configure a WebSphere MQ server bus member, the bus uses the MQ_INSTALL_ROOT setting to establish a direct binding to WebSphere MQ rather than a TCP/IP network connection.
5. At Cell scope or Node scope, set the WebSphere Application Server MQ_CLEAR_MQ_FROM_OSGI_CACHE_ON_SHUTDOWN environment variable to True. This allows application server startup to automatically take account of changes that are made to the MQ_INSTALL_ROOT environment variable and WebSphere MQ JMS client libraries while the application server is stopped. If you do not set this variable, you must restart the application server a second time after any changes of this type, to enable the application to perform messaging using the WebSphere MQ messaging provider.

* Configuring a unified connection factory for the WebSphere MQ messaging provider
1. In the navigation pane, click Resources > JMS > JMS providers.
2. Select the WebSphere MQ messaging provider that you want to support the destination.
3. Optional: Change the Scope setting to the level at which the connection factory is visible to applications.
4. In the contents pane, under Additional Properties, click Connection factories to display a table of existing unified connection factories, with a summary of their properties.
5. To view or change the properties of an existing unified connection factory, click its name in the list. Otherwise, to create a new connection factory, complete the following steps:
1. Click New in the content pane.
2. Specify the following required properties.
Name
The name by which this connection factory is known for administrative purposes within IBM® WebSphere Application Server.
JNDI name
The JNDI name that is used to bind the connection factory into the namespace.
CCSID
The coded character set identifier for use with the WebSphere MQ queue manager; for example: 850. See the properties for the unified JMS connection factory for more details.
3. Click Apply. This defines the JMS connection factory to WebSphere Application Server, and enables you to view or change additional properties.
6. Optional: Change properties for the unified JMS connection factory, according to your needs.
7. Optional: Change connection pool properties and session pool properties, according to your needs.
8. Configure a non-default SSL repertoire. To do this, specify a custom property on the connection factory with name ssl.repertoire and a value of the name of the repertoire alias to be used. A list of available repertoire aliases can be found by navigating to Security > SSL certificate and key management > SSL configurations . For WebSphere MQ the only settings in the SSL configuration that are used are:
* Location of the Trust store and its password
* Location of the Key store and its password
Information about supported SSL cipher suites is ignored by the WebSphere MQ client.
9. Click OK.
10. Save any changes to the master configuration.
11. To have the changed configuration take effect, stop then restart the application server.

* Configuring a queue connection factory for the WebSphere MQ messaging provider
1. In the navigation pane, click Resources > JMS > JMS providers.
2. Select the WebSphere MQ messaging provider for the queue connection factory that you want to configure.
3. Optional: Change the Scope setting to the level at which the connection factory is visible to applications.
4. In the contents pane, under Additional Properties, click Queue connection factories to display a table of existing queue connection factories, with a summary of their properties.
5. To view or change the properties of an existing queue connection factory, click its name in the list. Otherwise, to create a new connection factory, complete the following steps:
1. Click New in the content pane.
2. Specify the following required properties.
Name
The name by which this connection factory is known for administrative purposes within IBM® WebSphere Application Server.
JNDI name
The JNDI name that is used to bind the connection factory into the namespace.
CCSID
The coded character set identifier for use with the WebSphere MQ queue manager; for example: 850. See the properties for the queue connection factory for more details.
3. Click Apply. This defines the connection factory to WebSphere Application Server, and enables you to view or change additional properties.
6. Optional: Change properties for the queue connection factory, according to your needs.
7. Optional: Change connection pool properties and session pool properties, according to your needs.
8. Configure a non-default SSL repertoire. To do this, specify a custom property on the connection factory with name ssl.repertoire and a value of the name of the repertoire alias to be used. A list of available repertoire aliases can be found by navigating to Security > SSL certificate and key management > SSL configurations . For WebSphere MQ the only settings in the SSL configuration that are used are:
* Location of the Trust store and its password
* Location of the Key store and its password
Information about supported SSL cipher suites is ignored by the WebSphere MQ client.
9. Click OK.
10. Save any changes to the master configuration.
11. To have the changed configuration take effect, stop then restart the application server.

* Configuring a topic connection factory for the WebSphere MQ messaging provider
1. In the navigation pane, click Resources > JMS > JMS providers.
2. Select the WebSphere MQ messaging provider for the topic connection factory that you want to configure.
3. Optional: Change the Scope setting to the level at which the connection factory is visible to applications.
4. In the contents pane, under Additional Properties, click Topic connection factories to display a table listing any existing topic connection factories, with a summary of their properties.
5. To view or change the properties of an existing topic connection factory, click its name in the list. Otherwise, to create a new connection factory, complete the following steps:
1. Click New in the content pane.
2. Specify the following required properties. You can specify other properties, as described in a later step.
Name
The name by which this topic connection factory is known for administrative purposes within IBM® WebSphere Application Server.
JNDI Name
The JNDI name that is used to bind the topic connection factory into the name space.
CCSID
The coded character set identifier for use with the WebSphere MQ queue manager; for example: 850. See the properties for the topic connection factory for more details.
3. Click Apply. This defines the destination to WebSphere Application Server, and enables you to view or change additional properties.
6. Optional: Change properties for the topic connection factory, according to your needs.
7. Optional: Change connection pool properties and session pool properties, according to your needs.
8. Configure a non-default SSL repertoire. To do this, specify a custom property on the connection factory with name ssl.repertoire and a value of the name of the repertoire alias to be used. A list of available repertoire aliases can be found by navigating to Security > SSL certificate and key management > SSL configurations . For WebSphere MQ the only settings in the SSL configuration that are used are:
* Location of the Trust store and its password
* Location of the Key store and its password
Information about supported SSL cipher suites is ignored by the WebSphere MQ client.
9. Click OK.
10. Save any changes to the master configuration.
11. To have the changed configuration take effect, stop then restart the application server.

* Configuring a queue for the WebSphere MQ messaging provider
1. In the navigation pane, click Resources > JMS > JMS providers.
2. Select the WebSphere MQ messaging provider for the queue destination that you want to configure.
3. Optional: Change the Scope setting to the level at which the JMS destination is visible to applications.
4. In the contents pane, under Additional Properties, click Queues to display a table of existing queue destinations, with a summary of their properties.
5. To view or change the properties of an existing queue destination, click its name in the list. Otherwise, to create a new queue, complete the following steps:
1. Click New in the content pane.
2. Specify the following required properties.
Name
The name by which this queue destination is known for administrative purposes within IBM® WebSphere Application Server.
JNDI name
The JNDI name that is used to bind the queue destination into the namespace.
Base Queue Name
The name of the queue to which messages are sent, on the queue manager specified by the Base Queue Manager Name property.
CCSID
The coded character set identifier for use with the WebSphere MQ queue manager; for example: 850. See the properties for the queue destination for more details.
3. Click Apply. This defines the queue destination to WebSphere Application Server, and enables you to view or change additional properties.
6. Optional: Change properties for the queue destination, according to your needs.
7. Optional: If you want WebSphere Application Server to try to use the WebSphere MQ queue manager’s remote administration utilities to create the queue, configure the WebSphere MQ Queue Connection properties.
If you have already created your underlying queue in WebSphere MQ using its administration tools (such as runmqsc or MQ Explorer), you do not need to configure any of the WebSphere MQ Queue Connection properties. You only need to configure these properties if you want WebSphere Application Server to try to use the WebSphere MQ queue manager’s remote administration utilities to create the queue.
To be able to browse or change these MQ Config properties, you must have installed the WebSphere MQ client. If you have not done this, the administrative console displays messages like the following:
The WMQQueueDefiner MBean has encountered an error.
WMSG0331E: The MQ Client is required for this functionality, but it is not installed.
Note: For any changes to these properties to take effect on the queue manager, the WebSphere MQ Queue Manager on which the queue resides (or will reside) must be configured for remote administration and be running.
For more details about these properties, see WebSphere MQ config properties for the queue destination.
8. Click OK.
9. Save any changes to the master configuration.
10. To have the changed configuration take effect, stop then restart the application server.

* Configuring a topic for the WebSphere MQ messaging provider
1. In the navigation pane, click Resources > JMS > JMS providers.
2. Select the WebSphere MQ messaging provider for the topic destination that you want to configure.
3. Optional: Change the Scope setting to the level at which the topic destination is visible to applications.
4. In the contents pane, under Additional Properties, click Topics to display a table of existing topic destinations, with a summary of their properties.
5. To view or change the properties of an existing topic destination, click its name in the list. Otherwise, to create a new topic destination, complete the following steps:
1. Click New in the content pane.
2. Specify the following required properties. You can specify other properties, as described in a later step.
Name
The name by which this topic destination is known for administrative purposes within IBM® WebSphere Application Server.
JNDI Name
The JNDI name that is used to bind the topic destination into the namespace.
Base Topic Name
The name of the WebSphere MQ topic to which messages are sent.
CCSID
The coded character set identifier for use with the WebSphere MQ queue manager; for example: 850. See the properties for the topic destination for more details.
3. Click Apply. This defines the topic destination to WebSphere Application Server, and enables you to view or change additional properties.
6. Optional: Change properties for the topic destination, according to your needs.
7. Click OK.
8. Save any changes to the master configuration.
9. To have the changed configuration take effect, stop then restart the application server.

* Configuring WebSphere MQ connection pooling
1. Display the Message Listener Service properties for the application server
1. In the navigation pane, click Servers > Application Servers
2. In the content pane, click the name of the application server.
3. Under Additional Properties, click Message Listener Service properties.
2. Select Custom Properties, to enable WebSphere MQ connection pooling, add the following custom properties:
MQJMS.POOLING.THRESHOLD
The maximum number of unused connections in the pool.
MQJMS.POOLING.TIMEOUT
The timeout in milliseconds for unused connections in the pool.
3. Click OK.
4. Save any changes to the master configuration.
5. To have the changed configuration take effect, stop then restart the application server.

WAS + MQ

Follow the instruction

1. Service integration :make the BUS and Q destination point
2. Resource : make the Q connection factory and Q (and relate it with above value)
3. You can follow the Service integration Bus breadcrumb till you find the message in the Q pool.
(the message is readable)

To test use this:

1. install apps called Universal Test client (UTC.ear) at install-root\deploytool\itp\plugins\com.ibm.ws.ast.st.utc_6.1.6.v200811230003\IBMUTC.ear

this tool can be used to verify the Q in the pool.

WAS + TDS

From TDS:
A. installing the IBM TDS.
B. installing the WASembedded-IBMTDS
C. start the instance in the services.msc.
D. start the server1 in the TDS profile WAS.
E. go to TDS admin console : http://localhost:12100/IDSWebApp/IDSjsp/Login.jsp?showConsoleAdminLogin=true
F. default user:superadmin passwd: secret
G. Register the instance LDAP you want (ex. localhost 389)
H. Then you can manage this LDAP localhost 389 by the url
G. user: cn=...(usually root) passwd:.... (depends on what you put during the installation)
I. Go to server admininstration, add suffix ex. o=nehalorg
J. Go to directory management - manage entry : add o=nehalorg (parent leaves it empty) at the value you can put it same nehalorg (so there wudnt be multiple values).
K. From here you can create our own LDAP structure.
L. if you want to add an object (ex. class=person)and get "invalid dn entry". make sure in the relative dn you dint just put the calue, do like this "cn=value" (eihter it is cn or o or ou or whatever, you can see it right under it, there is one mandatory value, if it is cn then you put cn)


From WAS:
A. Security -> secure admin and infra -> available realm = standaolne ldap ==> set and then configure

B. primary admin user name =
a.either full DN (cn=wasuserPerson,ou=brno,o=nehalorg) or
b.the value (wasuserperson)
(only after you modify the "advance ldap setting" filter like cn=%v , notice %v is value).
At this time, WAS change the type to "custom ldap" instead of IBM TDS (uid=%v, objectclass=ePerson).

C. put the BaseDN (as the point to start search).
D. Bind DN is not necessary,it is only for anonymous connection.
E. restart java.
F. if you changed the "advance ldap setting" then you can use the value, instead of the long dn.
G. case doesnt sensitive.

NOTE:
1. SECJ0369E: Authentication failed when using LTPA. The exception is javax.naming.AuthenticationNotSupportedException: [LDAP: error code 48 - Inappropriate Authentication].
==> This means you use "user" which has no passwd attribute (it depends on the object class)

2. LdapRegistryI E SECJ0361E: Authentication failed for wasuserinet because user is not found in the registry.
==> if you have ensured that you use full dn, and this still appears, This means you need to configure the ldap filter search in "advance ldap setting".

Thursday 7 January 2010

java -Xdump:help

Usage:

-Xdump:help Print general dump help
-Xdump:none Ignore all previous/default dump options
-Xdump:events List available trigger events
-Xdump:request List additional VM requests
-Xdump:tokens List recognized label tokens
-Xdump:dynamic Enable support for pluggable agents
-Xdump:what Show registered agents on startup

-Xdump::help Print detailed dump help
-Xdump::none Ignore previous dump options of this type
-Xdump::defaults Print/update default settings for this type
-Xdump: Request this type of dump (using defaults)

Dump types:

-Xdump:console Basic thread dump to stderr
-Xdump:system Capture raw process image
-Xdump:tool Run command line program
-Xdump:java Write application summary
-Xdump:heap Capture raw heap image
-Xdump:snap Take a snap of the trace buffers

Example:

java -Xdump:heap:none -Xdump:heap:events=fullgc class [args...]

Turns off default heapdumps, then requests a heapdump on every full GC.


.............................

java -Xdump:java:help

Write application summary:

-Xdump:java[:defaults][:

Tuesday 5 January 2010

How to mount - format - record CD on linux

Firstly, generate iso file form the file(s)/folder(s) that you wanna write to disc:
CD can only accept iso file to be written into.

1. Make iso file from sourcefile(s)
mkisofs -r -R -J -l -L -allow-multidot -o /tmp/theisofilewillbe.iso -graft-points /folderwillbeindisc=/foldersource
Note: /folderwillbeindisc ; put it as "/" (without quote), to make it in top of the CD (the root)


2. Multisession Disc:
sudo cdrecord -v -eject speed=4 dev=1,0,0 -multi /tmp/theisofile.iso

Note: Wihtout multisession you can't add more files to the disc.

3. To add file(s) on multisession disc:
run : cdrecord -msinfo (If you get message like "busy" then please umount it first)
Using /dev/cdrom of unknown capabilities
0,23073
This is first sector and last sector!!

4. mkisofs -J -r -V SecondBurn -o cd-iso-image-file-2.iso -C 0,23073 -M 1,0,0 /file(s)folder
Note: there can't be same file name in one folder, so put -graft-point options or it would be failed though it seems working but the new sector would not be like the working one: 23073,42078 (almost double).

Note: at this step the CD must be in the dev (in this case M 1,0,0) or the new session can not be merged with the previous one. If you don't put the CD nor define -M options then when burn, it would be not there the result though the sector was correctly extended.

5. cdrecord -v -eject speed=4 dev=1,0,0 -multi isofile.iso
Note: if you don't see the correct result, try eject and insert the CD again.

6. To fallback the last burn : cdrecord -blank:session

7. The fast blank to blank whole CD, and re-burn again, usually works but re-eject afterward to get the wanted result.

Additional help:
A.To erase a CD-RW use the command : cdrecord -blank=disk
B. If you have multi CDROM, then:
To identify the DEV ID of your CDROM use 'cdrecord -scanbus'
After that use 'cdrecord dev=1,0,0 -blank=disk'
Note: dev=xxxx depends on which CDROM you wanna format/delete.

Help: cdrecord -blank=help


Troubleshoot:
a. How to mount:
sudo mount /dev/scd0
mount: block device /dev/sr0 is write-protected, mounting read-only
===> This is normal, noway you can make it as read-write.

b. If you mount and get additional message seems error like not known Filesystem, it is also normal, it means that the CD blank, and can't be mounted, so use cdrecord to write/burn something.

c. It will be mounted to a directory, as in /etc/fstab

d. you can check if it is mounted or not by type "mount" (or cat /etc/mtab) and see if any "cdrom" word there

e. dmesg | tail -f can be helpful to identify weird problem.

f. mount: wrong fs type, bad option, bad superblock on /dev/sr0,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so
===> Just re-eject the CD.

Got the detail on:
http://www.yolinux.com/TUTORIALS/LinuxTutorialCDBurn.html

Excerpt:
YoLinux Tutorial: Burning a CD or DVD
Using command line and GUI interfaces to burn/rip a CD or DVD.

Tux burning a CD It is typically easiest to use one of the GUI programs to burn a data or audio CD. These are typically graphical front ends to the command line programs cdrecord. My favorite for ease of use and creating / copying CD's is X-CdRoast.

Data CD's require that a file system image be generated prior to burning. This is typically the ISO-9660 CD standard. (And there are many non-standard variants and extensions.)

CD's can be burned in DAO (Disk At Once) or TAO (Track At Once) mode. The only reason to use DAO mode is when burning audio CD's. Audio CD's burned TAO will have 2 second gaps between tracks. If one is copying an entire CD, TAO mode will introduce these 2 second gaps if they do not already exist. DAO mode uses a raw CD access mode and can avoid these gaps between tracks.

DVDs allow for greater storage and also support additional formats for video.

This tutorial will cover audio and data CD and DVD copying, generation and burning software applications. Rip or burn a data backup CD or DVD on linux with ease.

Related YoLinux Tutorials:

°Linux Video

°Real Video Streaming

°Systems Administration

°Network configuration

°Internet Security

°Firefox Configuration / Plug-ins

°Ripping MP3s

°Streaming MP3s

°Integrate Linux into MS/Windows environment

°YoLinux Tutorials Index

Free Information Technology Magazines and Document Downloads
TradePub link image

Free Information Technology Magazine Subscriptions and Document Downloads

Bookmark and Share

Quantcast


Terms and definitions:

Term Description
CD-R CD-Recordable. Write Once, Read Many. (WORM)
CD-RW CD-Rewritable. Write Many, Read Many.
ISO-9660 DOS compatible file system (8.3 character file names) for CD's. Requires extensions like "Rock Ridge" to fully support UNIX/Linux
Rock Ridge ISO-9660 extension to allow long file names, UNIX symlinks and file protection (rwx). Windows readable.
Joliet Microsoft extension to ISO-9660. Creates MS/Windows 95 like 8.3 file names with translation to 64 character names.
Multisession Allows adding/writing files at a later time. Session must be closed before it is readable.
DAO Burn entire audio CD at once in a single burn. It's the only way to avoid delays between tracks
TAO Burn a track at a time. Results in 2 second delay between audio tracks.

GUI Front Ends for CD Burning:

* X-CdRoast:
o Home Page - Version 0.96e works well for data CD's. See configuration notes below for version 0.98 which has improved support for audio CD's.
o readme (0.96e)
Note: By installing X-CdRoast on Red Hat 9.0, you may develop conflicts with the archive CD burning facility within the Nautilus file manager. (Drag and drop files to the burn folder)
* GNOME Nautilus:
* gCombust: - Uses Gnome drag and drop interface.
* GNOME Toaster:
* Disc-O-Matic - CD/DVD archiving tool
* CDrecorder - GUI for cdrecord and mkisofs
* ECLipt Roaster - GTK interface to mkisofs and cdrecord for writing CDs on the fly.
* KisoCD - KDE CD burning front end to cdrecord and mkisofs
* SSCDRFE - Qt based Small Simple CDR front end
* TkBurn
* BurnIT - JAVA front-end to cdrecord, mksiofs and cdda2wav
* FireBurner - Burn data, audio, mixed mode, MP3's.
* Mp3Cd - Create audio CD from MP3's.

GNOME: Using the Nautilus desktop "file manager" to burn a data CD:

Within the Nautilus file manager is "CD Creator".

Note: Some systems require that you be root. Also see the YoLinux.com Managing groups tutorial to configure a system to grant users CD privileges.

File or directory backup:

* Launch "CD Creator" using one of the following methods:
1. Insert blank CD-R(W), Nautilus "CD Creator" will appear
2. Type the URL "burn:///" in Nautilus.
3. From the Nautilus toolbar select "Go" + "CD Creator".
* Select the "Home Folder" within the tree portion of the GUI
OR: launch a new instance of the Nautilus file manager by double clicking the "Home" icon on the desktop.
Select files and directories to back-up. Use ctrl key for multiple selections.
Right click to bring up an options menu and select "Copy File". (or Cut, ...)
* Select the "Blank CD-R Disk:" item in the "Tree"
OR: If using two Nautilus windows, drag files/directories to the "CD Creator" window.
Right click to bring up an options menu and select "Paste Files".
[Potential Pitfall]: When the blank CD is inserted into the drive, Nautilus should start with a CD showing no contents. If the contents from the last burn remail listed, select them and press the "delete" key to remove the listing.
* Select the "Write to Disc" button.
* This will launch a new dialog box "Write to Disc".
Select the "Write" button:
* This will burn a CD with the contents you dropped in the "Blank CD-R Disc" branch of the Tree menu.

CD/DVD system behavior configuration for the Gnome desktop:
Set system behavior using the Gnome command gnome-volume-properties

X-CdRoast 0.98 installation / configuration notes:

No special configuration is required for Red Hat 8.0/9.0. It just works.

The version of X-CdRoast (.98) supports burning audio CD's DAO. (Because cdrecord now supports it). This now makes it a good tool for copying both audio and data CD's.

X-CdRoast does not act alone. It is a GUI front-end to the Linux command line tools such as cdrecord. Thus it is necessary to download the RPM packages containing cdrecord, mkisofs, cdda2wav and the appropriate audio and graphics support libraries.

Red Hat 7.2:

The current version of the X-CdRoast RPM for Red Hat 7.2 does not use the sudo configuration as detailed below. Red Hat has configured X-CdRoast to run under consolehelper which requires the root password if operating as a non-root user. The X-CdRoast site has a different RPM for RH7.2 which does not use consolehelper. Newer versions of cdrecord, cdda2wav, ... are also included/incorporated in RH7.2 making life simple.

Red Hat 7.1 Download / Installation:

* cdrecord, devel, mkisofs, cdda2wav RPM's available from this site
Install using rpm command and include both cdrecord and cdrecord-devel on the command line to handle dependencies.
rpm -ivh cdrecord-1.9-1..........rpm cdrecord-devel-1.9-1..........rpm
* X-CdRoast 0.98 download page

Version 0.98 of X-CdRoast requires gtk+ configuration. Older versions of X-CdRoast (i.e. 0.96e) do not require this. The GTK+ toolkit had some changes in version 1.2.9 which require the attention of the administrator (set up sudo) as detailed below. (Explanation as to why this occurred)

Configuration:The following configuration will allow you to run X-CdRoast as root: (This and sudo configuration required for regular users - see below)

* Check for the version of GTK+ on your system:

rpm -q gtk+
gtk+-1.2.9-0_plain_1


Version 1.2.9 require these changes in order to make X-CdRoast work:

* Check for the sgid-bit on xcdrgtk:

ls -l /usr/bin/xcdrgtk

-rwxr-sr-x 1 root cdwrite 395216 Dec 3 07:12 /usr/bin/xcdrgtk


* As root, remove the sgid-bit on xcdrgtk:

chmod 755 /usr/bin/xcdrgtk

ls -l /usr/bin/xcdrgtk /usr/bin/xcdroast

-rwxr-xr-x 1 root cdwrite 395216 Dec 3 07:12 /usr/bin/xcdrgtk*
lrwxrwxrwx 1 root cdwrite 7 Apr 10 12:44 /usr/bin/xcdroast -> xcdrgtk


SUDO configuration so that regular users may run X-CdRoast: (sudo RPMs are available from the RedHat Power tools CD. [sudo Home page])

Use the command /usr/sbin/visudo (requires knowledge of vi or use another editor but return file /etc/sudoers to "-r--r-----" file access.)

File: /etc/sudoers:

user-name computer-hostname=NOPASSWD:/usr/bin/xcdroast

Run: sudo /usr/bin/xcdroast &
or use a desktop launcher with the command: cd $HOME; sudo /usr/bin/xcdroast

Links:

* sudo man page

Command line CD burning and support routines: Info, Downloads and Command line examples

* cdrecord: Home Page - man page - [Downloads] - Command line interface to burn CD's TAO and DAO.
* mkisofs: Home Page - man page - Build an ISO file system image. (as a file on hard drive)
* cdparanoia: Home Page - man page - audio CD reading utility
* cdrdao: Home Page - man page - Burn audio CDs without introducing 2 second gaps between tracks. Best for duplicating audio CDs. (Burns DAO-Disk At Once instead of TAO-Track At Once) Supports drives working in raw mode only. List of supported drives. Also has Xcdrdaow front-end.
* xcdrdao: - man page - GUI interface for cdrdao. Included in cdrdao RPM.

Downloads:

* cdrecord, mkisofs and cdda2wav RPM's available from this page
* cdrdao RPMS are available on the Red Hat Powertools CD. [RH6.2 RPM of cdrdao] or see rpmfind.net. - [Alternate download site]
This in turn requires gtk-- which is also available as an RPM on the Red Hat Powertools CD.

Getting Ready:

Note: Use the command cdrecord -scanbus to scan your SCSI bus to determine the device description. (i.e. 0,4,0). This will perform a scan and provide results for real SCSI systems or for the virtual SCSI subsystem Linux uses with IDE CD/DVD's.

Examples:

* SCSI CD system:

# cdrecord -scanbus
Cdrecord 1.8 (i686-pc-linux-gnu) Copyright (C) 1995-2000 J�rg Schilling
Using libscg version 'schily-0.1'
scsibus0:
0,0,0 0) 'SEAGATE ' 'ST36530W ' '1498' Disk
0,1,0 1) 'SEAGATE ' 'ST39173W ' '6244' Disk
0,2,0 2) *
0,3,0 3) *
0,4,0 4) 'HP ' 'CD-Writer+ 9200 ' '1.0c' Removable CD-ROM
0,5,0 5) *
0,6,0 6) *
0,7,0 7) *

* CD/DVD ATA system: This will report on your SCSI device name to use when burning a CD. In this case "ATA".

# cdrecord -scanbus
Cdrecord-Clone 2.01-dvd (--) Copyright (C) 1995-2004 J�rg Schilling
...
scsidev: 'ATA'
devname: 'ATA'
scsibus: -2 target: -2 lun: -2
Linux sg driver version: 3.5.27
Using libscg version 'schily-0.8'.
cdrecord: Warning: using unofficial libscg version (schily Red Hat-scsi-linux-sg.c-1.83-RH
scsibus1:
1,0,0 100) '_NEC ' 'DVD_RW ND-2510A ' '2.15' Removable CD-ROM
1,1,0 101) *
1,2,0 102) *
1,3,0 103) *
1,4,0 104) *
1,5,0 105) *
1,6,0 106) *
1,7,0 107) *

[Potential Pitfall]: You may have to be root to access the CD with the appropriate permissions.

Cdrecord 1.10 (i686-pc-linux-gnu) Copyright (C) 1995-2001 J?rg Schilling
cdrecord: Permission denied. Cannot open '/dev/sg0'. Cannot open SCSI driver.
cdrecord: For possible targets try 'cdrecord -scanbus'. Make sure you are root.

Note:

* Linux Kernel 2.2 and 2.4 use SCSI emulation when accessing an IDE/ATAPI CD drive. See "IDE and Linux Kernel considerations" below.
Example device reference: dev=0,4,0
* Linux kernel 2.6 allows one to directly access ATAPI. Prefix the device name with "ATAPI:".
Example device reference: dev=ATAPI:0,4,0
* To get a list of supported devices: cdrecord dev=HELP
i.e.: ATAPI, ATA, RSCSI (remote scsi)

YoLinux SCSI tutorial

Note:

* Data CD's must be mounted before one may access them. The CD may be mounted by:
o The automounter
OR
o The graphical mounting tool "Start" + "System" + "Disk Management" + "Mount"
OR
/usr/bin/usermount
OR
o The mount command:
+ Red Hat 7/8/9: mount -r -t iso9660 /dev/cdrom /mnt/cdrom
+ Fedora Core 3 (kernel 2.6): mount -r -t iso9660 /dev/hdc /media/cdrecorder
Note: Fedora Core 3 is using /media as a mount point for floppy drives, CD-ROMs and DVDs.

This command should work for a Red Hat installation. Other distributions may require the set-up as described in the YoLinux Sys Admin Tutorial - Mounting a CD.

Don't forget to un-mount the CD with umount /mnt/cdrom
[Potential Pitfall]: There is NO "N" in umount!!!

* It is often required that the user be root in order to perform these functions. Switch user using the command "su -" then enter the appropriate password.

CD burning: Command line examples

Run the CD burning commands as root as this is typically the only user on the system with privileges to burn a CD. One may also configure user groups and permissions to grant access for a user to burn a CD. See the YoLinux tutorial Managing Group Access.

Use the command "cdrecord" as described above to obtain entries for the "dev" and "--device" command line arguments.

Data CD's will require that you create the ISO image first (use command mkisofs) and then burn the ISO image onto the CD (use command cdrecord). See mkisofs instructions below.

Examples:

* If you download an ISO CD image from the net (i.e. a Linux CD) or possess a CD you wish to copy:
1. Create ISO image file:
o Copy a data CD (fast): cat /dev/scd0 > RedHat-7.0-i386-powertools.iso
or
o Mount CD to copy: mount -t iso9660 -o ro /dev/cdrom /mnt/cdrom
(Fedora core 3 (kernel 2.6): mount -t iso9660 -o ro /dev/cdrom /media/cdrecorder)
Create ISO file image copy of CD: mkisofs -r -o RedHat-7.0-i386-powertools.iso /mnt/cdrom

2. Burn ISO image onto CD:
o Linux 2.6 kernel: cdrecord -v -eject speed=16 dev=ATA:1,0,0 RedHat-7.0-i386-powertools.iso
o Linux 2.4 kernel: cdrecord -v speed=2 dev=0,4,0 RedHat-7.0-i386-powertools.iso

(Tip: Mounting the raw ISO file: mount -t iso9660 -o loop /home/user1/RedHat-9.0-i386-Disk1.iso /mnt/iso-1)

* Backup a directory to a CD:
o One step: mkisofs -r -R -J -l -L /home/user1 | cdrecord dev=0,4,0 -v --eject speed=4 -
o or the two step variation of the process:
1. mkisofs -r -R -J -l -L -allow-multidot -o /tmp/backup.iso -graft-points "/home/user1=/home/user1"
2. cdrecord dev=0,4,0 -v --eject speed=4
The "mkisofs" command is covered in detail below.
Make sure directory does not hold more than CD capacity (700Mb typically). Check with the command: du -s /home/user1

* Copy a data CD:
Mount CD to copy (kernel 2.4): mount -t iso9660 -o ro /dev/cdrom /mnt/cdrom
(Linux kernel 2.6: mount -t iso9660 -o ro /dev/cdrom /media/cdrecorder)
Create ISO file image copy of CD: mkisofs -r -R -J -l -L -o /tmp/cd-iso-image-file.iso /mnt/cdrom
Unmount: umount /mnt/cdrom (or umount /media/cdrecorder)
Load blank CD to burn.
Burn CD examples:
o Linux kernel 2.4: cdrecord -v speed=2 dev=0,4,0 -data /tmp/cd-iso-image-file.iso
(SCSI bus 0, device 4, lun 0)
o Linux kernel 2.6: cdrecord -v speed=2 dev=ATA:1,0,0 /tmp/cd-iso-image-file.iso
Note: The flag -data is default unless audio file extensions .au or .wav are used which defaults to -audio.

* Burn a multisession CD:
This is easiest with X-CdRoast which can perform the necessary calculations but it can also be done via the command line.
cdrecord -v -eject speed=4 dev=0,4,0 -multi cd-iso-image-file-1.iso
The CD is left in a mode which will allow it to receive more data.
To append data one must know where the last burn left off:
cdrecord dev=0,4,0 -msinfo
This will give you the starting and ending track numbers. i.e.: 0,16774. Use this result in mkisofs.
mkisofs -J -r -V SecondBurn -o cd-iso-image-file-2.iso -C 0,16774 -M 0,4,0 directory-with-files-to-add
The TOC from the first and second session will be combined.
Note: Omit the option "-multi on the final burn.

* Copy then burn single audio tracks TAO:
cdda2wav -D 0,4,0 -B /tmp/prefix-of-music-files
The files will be written out as /tmp/prefix-of-music-files_01.inf /tmp/prefix-of-music-files_01.wav /tmp/prefix-of-music-files_02.inf ...
o Linux 2.6 kernel: cdrecord -v speed=2 dev=ATA:1,0,0 -audio /tmp/prefix-of-music-files*.wav
o Linux 2.4 kernel: cdrecord -v speed=2 dev=0,4,0 -audio /tmp/prefix-of-music-files*.wav

* Burn audio tracks DAO:
Copy to hard drive:
cdda2wav -D 0,4,0 -g -O wav -S 1 -v30 -P 0 -n 75 -B /tmp/CD/track-01.wav /tmp/CD/track-02.wav ...
Burn CD:
o Linux 2.6 kernel: cdrecord dev=ATA:1,0,0 fs=4096k -v -useinfo speed=1 -dao -eject -pad -audio "/tmp/CD/track-01.wav" ...
o Linux 2.4 kernel: cdrecord dev=0,4,0 fs=4096k -v -useinfo speed=1 -dao -eject -pad -audio "/tmp/CD/track-01.wav" ...

* Burn WAV files:
o Linux 2.6 kernel: cdrecord dev=ATA:1,0,0 -eject speed=16 -pad -audio *.wav
o Linux 2.4 kernel: cdrecord dev=0,4,0 -eject speed=2 -pad -audio *.wav

* Copy then burn audio CD's DAO (two drives): See cdrdao examples
Copy the CD from the reader (SCSI 3) and burn on the Read/Writer (SCSI 4):
cdrdao copy --source-device 0,3,0 --device 0,4,0 --buffers 64
OR
cdrdao copy --source-device 0,3,0 --source-driver generic-mmc-raw --device 0,4,0 --driver generic-mmc-raw --buffers 64 toc-file.toc

* Copy audio CD DAO with one CD drive:
cdrdao copy --device 0,4,0 --buffers 64 --driver generic-mmc-raw --read-raw toc-file.toc
This will prompt you to insert the CD-R after an image of the source CD was created. ("Please insert a recordable medium and hit enter") The image file with name "cddata.bin" will be created in the current working directory. (file removed when done)

* Extract audio image from CD:
cdrdao read-cd --device 0,4,0 --buffers 64 --driver generic-mmc-raw --read-raw toc-file.toc
Creates file data.bin and table of contents file toc-file.toc. This is NOT a fast process.

Notes on CD burning:

* Audio CD's do not have a file system thus they cannot be mounted, just read.

* ISO 9660 refers to a data CD storage format and not to the standard audio CD.

* Don't run any applications while burning as this can lead to buffer under-run condition if your PC was not able to allocate enough time to processing the burn. This will then create a useless CD.

* I use a SCSI CD but for those that are using IDE you may be required to add the following entry to the end of /etc/lilo.conf and execute lilo -v. This allows linux to use a SCSI interface to the IDE CD drive.
append = "hdb=ide-scsi" (loads kernel loadable module: /lib/modules/2.X.XX-XX/kernel/drivers/scsi/ide-scsi.o)
OR
at the lilo boot prompt enter linux hdb=ide-scsi

* Want to hear that wav file to make sure it is the right song?
play file_01.wav

* Assign a higher priority to the process burning the CD: nice -18 cdrecord ...
Where the priority range is 19 (lowest) to -20 (highest). Default is 10.

* Device defaults can be found in the configuration file: /etc/cdrecord.conf
The default speed can be set: CDR_SPEED=4

Re-Writing a CD-RW:

The re-writable CD can be "blanked" before re-burning. (optional)

* Using XCdRoast GUI:
o "Create CD"
o "Write Tracks"
o "Blank CD-RW"
The CD is now ready to be re-inserted for burning.

* Command to "blank" a CD-RW:
o Fast blanking. Data is not overwritten:

cdrecord dev="1,0,0" -v -eject speed=4 blank=fast


o Full blanking. Data is overwritten and entire media is erased:

cdrecord dev="1,0,0" -v -eject speed=4 blank=all


where dev is the device as defined by "cdrecord -scanbus" as discussed above.

mkisofs: Creating an ISO image file (Creating Data CD image)

An ISO 9660 standard CD is a mountable file system. One may first generate an ISO image of the data files and directories on the hard drive before burning the CD. Once this is generated, it can be burned directly onto the CD. This is the standard used for data CD's and NOT the standard used for your typical CD player. The mkisofs command is used to generate an ISO image. The command cdrecord can then be used to burn an ISO image onto the CD.

Examples:

* Copy a directory tree from the hard drive into an ISO-9660 image:
(for the preferred Linux backup see the next option.)
mkisofs -r -o /fully-qualified-file-name-of-iso-image.iso /name-of-directory-containing-files-to-format-into-iso-image

* Copy an existing CDROM into a new ISO-9660 image:
mkisofs -r -o /fully-qualified-file-name-of-iso-image.iso /dev/cdrom

* Copy a directory tree from the hard drive into an ISO image (but probably not compatible with MS DOS): (Better for Linux backup as hidden files beginning with "." and long file names are preserved.)
mkisofs -r -R -J -l -L -o /fully-qualified-file-name-of-iso-image.iso /name-of-directory-containing-files-to-format-into-iso-image

* Example of command used to make Red Hat Power Tools CD:
mkisofs -A Red Hat Powertools/i386 7.1 -V Red Hat Powertools/i386 7.1 -r -R -J -v -T -x ./lost+found -o /ISO/Powertools-7.1-i386.iso

* Example of command to create Red Hat boot/install disk:
mkisofs -A Red Hat Linux/i386 7.1 -V Red Hat Linux/i386 7.1 -J -r -R -v -T -x ./lost+found -o /ISO/qa0408.4-i386-disc1.iso -b dosutils/autoboot/cdboot.img -c boot.cat

* Write Multiple directories to CD and place in a root directory of the assigned name: (Command is all on one line.)
mkisofs -r -R -J -l -L -o /ISO/MyBackupCD-20031214.iso -graft-points "/=/home/user1/DirectoryNameA" "/=/home/user1/DirectoryNameB/SubDirectoryNameC" "/=/home/user1/DirectoryNameD/FileName_1" "/=LocalDirectoryNameE"

This will create a CD ISO image with the following directory structure:

/FilesFromDirectoryNameA_1
/FilesFromDirectoryNameA_2
...
/SubDirectoryFromDirectoryNameA/...
/FilesAndDirectoriesFromDirectoryNameC/...
/FileName_1
/FileFromLocalDirectoryNameE_1
/FileFromLocalDirectoryNameE_2
/FileFromLocalDirectoryNameE_3
/...


Note:
o The last directory specified in the path name is NOT put on the CD but it's contents are.
o The reference to LocalDirectoryNameE is the directory which can be listed from the directory where the mkisofs command is issued.
o Double check what you created by mounting the ISO image (see below) and viewing the files.

* Specify root directory name of copied directories: (Command is all on one line.)
mkisofs -r -R -J -l -L -o /ISO/MyBackupCD-20031214.iso -graft-points "/DirectoryNameA=/home/user1/DirectoryNameA" "/SubDirectoryNameC=/home/user1/DirectoryNameB/SubDirectoryNameC"

This will create a CD ISO image with the following directory structure:

/DirectoryNameA/...
/SubDirectoryNameC/...


Note:
o When referencing a local directory sometimes one must end the CD directory name with a closing "/" in order to have the directory created on the CD. This was true with older versions (RH 8.0 but not 9.0) of mkisoffs:
-graft-points "/CD-dirname/=HardDriveLocal-dirname"

* The last directory specified in the path name is NOT put on the CD but it's contents are.

mkisofs options:

Command Option Description
-b Specifies the path and filename of the boot image to be used when making an "El Torito" bootable CD.
The pathname must be relative to the source path specified to mkisofs.
Required to make a bootable CD. The boot image must be exactly the size of either a 1.2, 1.44, or a 2.88 meg floppy.
-c Specifies the path and filename of the boot catalog to be used when making a bootable CD.
The pathname must be relative to the source path specified to mkisofs.
-l Allow 31 character file names. CD will not be 8.3 DOS compatible. (Who cares! Thus use this option.)
-o Name of iso9660 output image file
-r Set file permissions on CD to be publicly readable
-v Verbose execution.
-x Exclude given path from being written to CDROM.
-A Text string that will be written into the volume header
-J Generate Joliet directory records in addition to regular iso9660 file names.
This preserves the case (upper/lower) of file and directory names. Useful for use with MS/Windows environments.
-L
-allow-leading-dots Allows files to begin with "." (period). Use this option to write Unix/Linux hidden files. CD will not be DOS compatible. (Who cares! Thus use this option.)
-allow-multidot Allows files to contain multiple "."s (periods). Use this option to write Unix/Linux files which may contain more than one period. CD will not be DOS compatible. (Who cares! Thus use this option.)
-R Add Rock Ridge records to further describe the files. Preserves filename length and case.
-V Specifies the volume ID to be written into the master block.
-T Generate a file TRANS.TBL in each directory.
-graft-points Translate a directory to the root ("/") of the CD. See examples. (i.e.: -graft-points "/=DirectoryName/")
-pad Default (not necessary to specify this option). Pads the end of the whole image by 150 sectors (300 kB). Avoids read ahead errors on last file.

More info:

* mkisofs Home Page
* mkisofs man page

Rip it: Getting the data from an existing CD

* Copy all the data from a CD/DVD: cat /dev/scd0 > data.iso
Note: This will not pick up the MBR from a bootable installation CD or DVD.
* Copy all the data from a CD/DVD: dd if=/dev/scd0 of=/tmp/isofile.iso
Note: This will capture boot information from a bootable CD or DVD.
* Create ISO file image copy of CD: mkisofs -r -R -J -l -L -o /tmp/cd-iso-image-file.iso /mnt/cdrom
Note: The CD must be mounted first.

Also see other examples including ripping audio CDs (above) in this tutorial.

Mounting an ISO file as a file system:

After you have created an ISO 9660 image file, one can verify it's contents by mounting the raw ISO file as a file system.

mount -t iso9660 -o loop /home/user1/RedHat-9.0-i386-Disk1.iso /mnt/iso-1

This can also be mounted using an entry in the file: /etc/fstab

/home/user1/RedHat-9.0-i386-Disk1.iso /mnt/iso-1 iso9660 loop,ro 0 3

Gathering CD information: isoinfo

The isoinfo utility can be used to gather information in a ISO file or on a CD and verify the integrity of an iso9660 image.

Command Option Description
-h Help and summary of options
-d Primary volume descriptor (PVD) of iso9660 image.
-f Use find output formatting
-i /path-of-image
or
dev=/dev/cdrom Path of image to examine
or
specify device (one or the other, not both)
-p Output path table information
-R Print permissions, file names and ownerships (Rock Ridge extensions)
-J Print file names (Joliet extensions)
-T sector Print info about multi-session images.

Examples:

* List contents of an iso file: isoinfo -f -R -i isoFile.iso

* List info about a CD: isoinfo -d -i /dev/cdrom

CD-ROM is in ISO 9660 format
System id: LINUX
Volume id: FC/3 x86_64
Volume set id:
Publisher id:
Data preparer id:
Application id: FC/3 x86_64
Copyright File id:
Abstract File id:
Bibliographic File id:
Volume set size is: 1
Volume set sequence number is: 1
Logical block size is: 2048
Volume size is: 326239
El Torito VD version 1 found, boot catalog is in sector 123
Joliet with UCS level 3 found
Rock Ridge signatures version 1 found
Eltorito validation header:
Hid 1
Arch 0 (x86)
ID ''
Key 55 AA
Eltorito defaultboot header:
Bootid 88 (bootable)
Boot media 0 (No Emulation Boot)
Load segment 0
Sys type 0
Nsect 4
Bootoff 7C 124


Note: The block and volume size can be used as input to the following command to:
o Find the check sum of the CD:
dd if=/dev/cdrom bs=2048 count=326239 conv=notrunc,noerror | md5sum
(Linux 2.6 kernel 2.6: /media/cdrecorder)
o Create an iso image:
dd if=/dev/cdrom bs=2048 count=326239 conv=notrunc,noerror > file-name.iso

Man Pages:

* isoinfo man page
* dd man page

cdrdao: (DAO) Creating Audio CD's

The "Disc at once" (DAO) feature allows one to burn an audio CD without the two second gap between tracks. This is most useful when copying CDs as it allows one to burn the CD to be just like the original. The cdrdao command uses a "table of contents" (TOC) file as shown below.

Usage: cdrdao command [options] toc-file

Example: cdrdao write toc-file.toc

command Description
show-toc Prints out table of contents (toc) then exits
toc-info Prints out short toc-file summary
toc-size Prints total number of blocks for toc
read-toc Create toc file from audio CD
read-cd Create toc and rip audio data from CD
show-data Prints out audio data and exits
read-test Reads all audio files and exits
disk-info Shows information about inserted medium
unlock Unlock drive after failed writing
simulate Shortcut for 'write --simulate'
write Writes CD
copy Copies CD

option Description
--device x,y,z
SCSI bus, device, lun
i.e. 0,4,0 Sets SCSI device of CD-writer
--source-device x,y,z sets SCSI device of CD-ROM reader
(used by 'copy')
--driver driver-id Force usage of specified driver
--source-driver driver-id Force usage of specified driver for source device
(used by 'copy')
--simulate Perform a write simulation
--speed CD-speed Set writing speed
--multi Session will not be not closed
--eject Ejects cd after writing or simulation
--swap Swap byte order of audio files
--on-the-fly No image file is created. Perform copy in process.
--datafile file-name Name of toc file
Used by read-toc, read-cd and copy
--buffers number Sets fifo buffer size (min. 10, def 32)
--session number Selects session for read-toc/read-cd
--fast-toc Do not extract pre-gaps and index marks
--read-raw Read raw sectors for read-cd
--paranoia-mode number DAE paranoia mode (0..3)
--reload Reload the disk if necessary for writing
--force Force execution of operation
-v verbose-level Set verbose level
-n No pause before writing

See cdrdao examples and how to generate a toc file to control gaps (or lack thereof) during CD burn.

Create toc (table of contents file) which shows(read)/determines(write) the gaps in tracks.
cdrdao read-toc --driver generic-mmc-raw --read-raw --device 0,4,0 toc-file.toc

Sample TOC (Table Of Contents) file:

CD_DA

CATALOG "0075590506829"

// Track 1
TRACK AUDIO
NO COPY
NO PRE_EMPHASIS
TWO_CHANNEL_AUDIO
SILENCE 00:01:00
FILE "data.wav" 0 03:28:70
START 00:01:00


// Track 2
TRACK AUDIO
NO COPY
NO PRE_EMPHASIS
TWO_CHANNEL_AUDIO
FILE "data.wav" 03:28:70 02:14:45
START 00:04:52

...
..

// Track 10
TRACK AUDIO
NO COPY
NO PRE_EMPHASIS
TWO_CHANNEL_AUDIO
FILE "data.wav" 26:05:40 05:03:37


// Track 11
TRACK AUDIO
NO COPY
NO PRE_EMPHASIS
TWO_CHANNEL_AUDIO
FILE "data.wav" 31:09:02 04:50:50

Minimal example:

CD_DA

TRACK AUDIO
AUDIOFILE "song1.wav" 0

TRACK AUDIO
AUDIOFILE "song2.wav" 0

TRACK AUDIO
AUDIOFILE "song3.wav" 0

...
..

Where the "0" refers to the beginning of the file

Links:

* Sourceforge: cdrdao Home Page
* GedMaster - GNOME CDRDAO GUI

The IDE device interface and Linux Kernel considerations:

Most kernels as released by the various Linux distributions include SCSI emulation support and thus support for the IDE CD drive. Check this with the following command: dmesg | grep -i scsi

SCSI subsystem initialized
scsi0 : sata_via
scsi1 : sata_via

This shows that the SCSI subsystem is working. (example: Fedora Core 3)

The Linux kernel 2.6 (RH Fedora Core) can use ATAPI directly by prefixing "ATAPI:" (or for ATA devices, ATA:) to the device argument.
If using Linux kernel 2.2 or 2.4, SCSI emulation is required. The kernel module ide-scsi enables this support.

The Linux CD interface uses SCSI as a hardware interface with Linux kernel 2.2 or 2.4. If you have an IDE/ATAPI CD and Linux kernel 2.2 or 2.4, you must use Linux SCSI emulation. Compile this in as part of your kernel by adding the following support using "make xconfig":

* Block devices -> Include IDE/ATAPI CDROM support = n
* Block devices -> SCSI emulation support = y

Both SCSI support and SCSI CDROM support will be enabled. (y) not (M).
Support for SCSI controller card support should be disabled.

Linux kernel configuration parameters:

Kernel config parameter Description
CONFIG_ISO9660_FS=y ISO CD-ROM file system support.
CONFIG_BLK_DEV_IDE=y
CONFIG_BLK_DEV_IDECD=y IDE/ATAPI CD-ROM device support.
CONFIG_SCSI=y
CONFIG_CHR_DEV_SG=y
CONFIG_SCSI_device-specific-driver=y SCSI CD-ROM device support. (generic and driver specific)
CONFIG_UDF_FS=y DVD file system support.
CONFIG_UDF_RW=y Read/Write DVD file system.

See YoLinux Optimization: Compiling the Kernel

The SCSI driver is loaded when the kernel boots and is held in:

* Fedora Core 3: /etc/modprobe.conf
* Red Hat 7/8/9: /etc/modules.conf
* Red Hat (or for older systems) 5: /etc/conf.modules
* Debian /etc/modules

Example for a system actually equipped with a SCSI card:

alias scsi_hostadapter aic7xxx

Examples for IDE systems:

alias scsi_hostadapter sata_via
or
alias scsi_hostadapter ide-scsi

The module can be loaded manually: insmod ide-scsi
and
removed manually: rmmod ide-scsi

On systems with an IDE/ATAPI CD-Rom, the system will be emulating SCSI for interfacing with the CD-ROM. (kernel 2.2, 2.4) If the CD was device /dev/hdb (example) it will now be the first CD-ROM SCSI device /dev/scd0. The emulation will now let your IDE CD-ROM act like a SCSI device. The old /dev/cdrom soft link should now be re-linked to the new device.

rm /dev/cdrom
ln -s /dev/scd0 /dev/cdrom

If using the lilo boot loader add the following line: (for CD device /dev/hdb)
append=" hdb=ide-scsi"

The module may be loaded manually with the command: insmod ide-scsi

You know you are successful when cdrecord -scanbus recognizes your CD-ROM drive.

Related Linux CD Commands:

* autorun - automatically mounts/unmounts CDROMs and executes /path/to/cdrom/autorun after mounting
* cdda2wav - a sampling utility that dumps CD audio data into wav sound files
* hpcdtoppm - convert a Photo-CD file into a portable pixmapmcd
* ascd - An AfterStep and WindowMaker CD player
* xgetfile - X based audio cd player for cdrom drives
* xplaycd - X based audio cd player for cdrom drives
* xcdroast - CD-Writer Software

CD HowTOs:

* CD-Writing-HOWTO
* CDROM-HOWTO
* CD-ROM Burning
* CD Writing Guide

More CD Information:

* CD Links
* Linux CD links - CD building project
* FreeDB.org - The free alternative to CDDB
* CDDB: The motherload of CD databases
* xmcd: Motif based CD player with CDDB access.
Download/install: mkdir XMCD; cd XMCD; -download then-> tar xzf xmcdbin.tar.gz; su; chmod +x install.sh; ./install.sh
Config file: $HOME/.xmcdcfg/XMcd and /usr/X11R6/lib/X11/xmcd/app-defaults/XMcd
cron script to update local db: /usr/lib/X11/xmcd/scripts/genidx (Copy to /etc/cron.daily/)
Run: /usr/X11R6/bin/xmcd
* Internet Cover Art Index - CD and alblum cover artwork
* CdIndex.org - 1st generation - becomes MusicBrainz.org
* MusicBrainz.org - CdIndex 2nd generation.
* CdChanger Jukebox application description - [Download]
* DiskMakers.com - Equipment for mass duplication, supplies.
* O'Reily: Creating Audio CD content
* DiscMarket.com - CD/DVD manufacturing supplies.

List of CDDB servers:

freedb.freedb.org cddbp 8880 Requires no fees or registration
us.cddb.com cddbp 8880 Random US site
sc.ca.us.cddb.com cddbp 8880 Santa Clara, CA US
sc2.ca.us.cddb.com cddbp 8880 Santa Clara, CA US
sj.ca.us.cddb.com cddbp 8880 San Jose, CA US
sj2.ca.us.cddb.com cddbp 8880 San Jose, CA US

Software to generate CD covers:

* CdLabelgen: home page - Generated postscript files. Integrates with gcombust CD burner software.
o Web Interface - Create online!
* Disc-Cover: home page - Audio CD's (Plain looking covers)

Burn Video CD (VCD):

* about.com: Making VCDs (Video CD's) with Linux - by Aron Hsiao

CD standards:

* Digital Audio CD-DA (Compact Disk Digital Audio) - "Red Book":
o Philips standard
* CD-R, CD-RW, Magneto-optical CD-MO and Write Once CD-WO - "Orange Book":
o Description
o Philips info
* ISO 9660 CD-ROM "High Sierra" - "Yellow Book":
o ECMA-119
o ECMA-130 - CD-ROM XA (Extended)
* Photo CD (Kodak) - "Beige Book":
o Info
* "Rock Ridge" CD-ROM Unix":
o IEEE Rock Ridge Interchange protocol (pdf)
* Video CD vCD - "White Book":
o Super Video CD Overview
o MpegTV FAQ

Burn a DVD: (using growisofs)

DVDs allow for greater storage capacity. They are also different from CDs in that they must first be formatted. The growisofs command will perform this task automatically if it detects that the DVD is unformatted. The DVDs in these examples are burned DAO (Disk At Once).

Examples:

* Copy a simple video DVD with Linux:
1. Rip: Create the ISO image with mkisofs as detailed above.
Copy a DVD and create ISO image file: (Paths reflect Fedora Core 3 / Linux kernel 2.6 mount point: /media/cdrecorder)
Examples:
o mkisofs -pad -J -R -oCourtneyDVD.iso -graft-points "/AUDIO_TS=/media/cdrecorder/AUDIO_TS" "/VIDEO_TS=/media/cdrecorder/VIDEO_TS"
o mkisofs -dvd-video -oCourtneyDVD.iso -graft-points "/AUDIO_TS=/media/cdrecorder/AUDIO_TS" "/VIDEO_TS=/media/cdrecorder/VIDEO_TS"
(Use option -dvd-video to generate a DVD-Video compliant UDF file system. This will sort files appropriately and add padding. File video names must be in upper case.)

2. Burn: Burn DVD with Linux and the growisofs command. Since growisofs is also a front-end to mkisofs, one can append mkisofs commands:
Examples:
o growisofs -dvd-compat -Z /dev/dvd=CourtneyDVD.iso -speed=2
Note: /dev/dvd and /dev/dvdwriter are soft links to /dev/hdc
o Example of burning DVD using "dvdrecord":
+ Linux 2.6 kernel: dvdrecord speed=1 -dao dev=ATA:1,0,0 CourtneyDVD.iso
+ Linux 2.4 kernel: dvdrecord speed=1 -dao dev=1,0,0 CourtneyDVD.iso

* Blanking a rewritable DVD: growisofs -Z /dev/dvd=/dev/zero
Not required before burning but erases sensitive data.

* Burning a data DVD: growisofs -dvd-compat -speed=2 -Z /dev/dvd -R -J -pad /dir-path/file1 /dir-path/file2

* Burn data file to DVD: growisofs -Z /dev/dvd -R -J /dir/file-name

* Burning a data backup DVD using more mkisofs commands: (works on DVD+R and double layer DVD+R DL - Fedora Core 3)
o Example 1: growisofs -speed=8 -Z /dev/dvd -R -J -allow-multidot -allow-leading-dots -l -pad -graft-points "/home/user1=/home/user1"
(may not be DOS compatable because of long file name support, possible leading dot or multidot file names, but do you really care?)
o Example 2: growisofs -dvd-compat speed=2 -Z /dev/dvd -R -J -pad -graft-points "/DirectoryNameA=/home/user1/DirectoryNameA" "/SubDirectoryNameC=/home/user1/DirectoryNameB/SubDirectoryNameC"
(write a DVD which MS/Windows can read - but not MS/DOS due to the "-l" option and long file names)

Usage: growisofs [growisofs-options] /dev/dvd [mkisofs-options]

growisofs command line options:

Command Description
-version List version of program growisofs.
-Z /dev/dvd Burn initial session to specified path. Closes DVD and no additions can be made. You can specify -Z or -M but not both.
-M /dev/dvd Merge a new session to an existing one at the specified path. Multi-session. You can specify -Z or -M but not both.
-dvd-compat Write with DVD compatibility. Write once disks are closed at the end of the session.
-dry-run Perform steps but do not burn DVD. Checks for overburn condition (too much data for DVD). 4.7 Gbyte (4.7 x 10003) DVDs actually hold 4.377 Gb (4.3 x 10243) as measured by the computer system.
-overburn Don't use this. Turns off overburn condition checking.
-speed=1 Specify recording velocity. Recommend 1 or 2. Not as reliable as CDs for high speed burning.

Note that the -R (Rock Ridge) and -J (Joliet) are mkisofs commands as described above which are appended to growisofs.

Formatting is not required as it is performed by the growisofs command if required. A DVD can be re-formatted using the command dvd+rw-format or to "erase" data.

[Potential Pitfall]: The automounter will detect a formatted DVD and mount it. You can not burn a mounted DVD. To disable the automounter, make sure the mount option "noauto" is used in the file /etc/fstab for the mount point /media/cdrecorder

/dev/hdc /media/cdrecorder auto pamconsole,exec,noauto, ... 0 0

fstab man page

[Potential Pitfall]: Do NOT use the mkisofs option "-o" as this is unsupported with growisofs. There is no option to generate a file, you will be burning a DVD.

[Potential Pitfall]: If you hit the "memorylocked" resource limit and get the following error when issuing the growisofs (V6.0) command:

"unable to anonymously mmap 33554432: Resource temporarily unavailable"

Issue the following bash shell command: ulimit -l unlimited

[Potential Pitfall]: If using the Linux 2.4 kernel (i.e. 2.4.18), you will have to apply kernel patches to get full DVD support. The 2.6 kernel (i.e. Fedora Core 3) has good built-in DVD support.

DVD Media Tools:

The following tools are included in the Fedora Core RPM package dvd+rw-tools:

* growisofs: Burn DVD. See tutorial above.

* dvd+rw-format: Format DVD. Also performed by growisofs if needed.
Format a DVD: dvd+rw-format -lead-out /dev/dvd

Usage: dvd+rw-format [-force[=full]] [-lead-out|-blank[=full]] [-ssa[=none|default|max]] /dev/dvd

* dvd+rw-booktype: Utility for manipulating Book Type Field of Physical Format Descriptor located in lead-in of DVD+RW media.

Usage: dvd+rw-booktype [-dvd-rom-spec|-dvd+rw-spec|-dvd+r-spec|-inq] [-media|-unit|-unit+rw|-unit+r] /dev/dvd

* dvd+rw-mediainfo: Information about mounted DVD media. Note: This never worked for me.

Usage: dvd+rw-mediainfo /dev/dvd

* dvd-ram-control

Usage: dvd-ram-control [-arre] [-awre] [-rdonly|-rdwr] /dev/dvd

DVD Links:

DVD Tools:

* Home page: dvd+rw-tools - [Download]
* growisofs.c - Source code.
* Burn a DVD with X-CD-Roast
* K3B CD/DVD Burner

DVD HowTos:

* DVD ripping and transcoding
o transcode: utilities for transcoding video and audio codecs.
* Create video DVD from an AVI:
o Create video DVD from an AVI - Tutorial
o MJPEG tools [Download]
(Linux/Intel only due to use of Intels MMX instructions)
* Encoding with MPlayer / MEncoder

DVD Standards:

* Description of DVD video file format (overview)
* ECMA-267: DVD Read Only disk
* ECMA-272: DVD Rewritable disk
* ECMA-274: Data interchange using +RW format

The eject command:

This command allows one to open and close the CD tray as well as control a multi-disk changer. Note that the first disk in a changer is referred to as disk "0", the second as disk "1", etc...

eject man page

Samples:

* Open tray (eject): eject /dev/cdrom
(Fedora core 3 (kernel 2.6): eject /media/cdrecorder)
* Open tray (eject) default CD device: eject
* Close tray: eject -t /dev/cdrom
* Rotate changer to second CD: eject -c 1 /dev/cdrom

There is rumor of a device driver for /dev/changer but I have no info on this.

YoLinux.com Home Page
YoLinux Tutorial Index
Privacy Policy | Advertise with us | Feedback Form |
Unauthorized copying or redistribution prohibited.

Bookmark and Share
Quantcast

Copyright © 2001 - 2008 by Greg Ippolito