Thursday, August 7, 2008

Log4j writing to a dynamic log file for every run

In my recent project, I was using Log4J and I had a requirement where I had to write to a new dynamic log file every time and the name of the log file was determined at runtime. In specific terms each run would produce a Project and the log file had to reflect the project name. Since this project name could be repeated across runs, the timestamp needed to be added to the log file. I searched across Google and didn't find much help in this regard. So I decided to post the code I wrote.

I didn't want to lose setting the log levels from the log4j.xml file and wanted all the options I could configure for the FileAppender except the file name. The file name was configured too, but the code had to overwrite and create a new file at runtime. Here is the code..



Date projDate = new Date(Long.parseLong(project.getTimeStamp()));
StringBuffer dateStr = new StringBuffer();
SimpleDateFormat sdf = new SimpleDateFormat("yyyy_MM_dd_HH_mm_ss");
dateStr = sdf.format(projDate, dateStr, new FieldPosition(0));
String logFileName = logsDir + File.separator + project.getName() + "_" + dateStr.toString() + ".log";
log.info("\n**************Log file for this run: " + logFileName + "\n**************\n");

Logger rootLogger = Logger.getRootLogger();
Enumeration appenders = rootLogger.getAllAppenders();
FileAppender fa = null;
while(appenders.hasMoreElements())
{
Appender currAppender = (Appender) appenders.nextElement();
if(currAppender instanceof FileAppender)
{
fa = (FileAppender) currAppender;
}
}
if(fa != null)
{
fa.setFile(logFileName);
fa.activateOptions();
}
else
{
log.info("No File Appender found");
}



That's it. The logs were created for each and parallel runs would write to different files...

19 comments:

Anonymous said...

Very nice. Thanks for posting this!

Karthikeyan C said...

Can you please explain "What is run"?
Thanks for your time involved.

Saagar said...

@ckarthi
run as in for each run of the java program, we needed a separate log file with a dynamic name..

Andy Y said...

Why not use a custom appender? I've done this a few times in the past to generate log files based on runtime parameters. Just extend an existing appender like RollingFileAppender and during construction set the filename to whatever dynamic parameter it has to be.

Then reference it as a target appender in your log4j configuration file & problem solved :).

This does leave a problem that somehow you need to define your dynamic runtime parameter to change the log file name with. I opted for a static variable. Since this is a production only appender the tight coupling it caused I felt was the lesser of two evils.

SARA said...

Log file for this run: c:\PROJECT\_2008_08_11_12_04_08.log

I got the following error.

No File Appender found

help me to solve this error.

Saagar said...

@handlename
Did you define your FileAppender in the log4j.xml or the log4j.properties file? You need to define one and the custom code overwrites that..

Anonymous said...

Hi Saagar,

I tried your code and used RollingFileAppender instead , because that is our requirement, but the problem we are facing is the log message is being printed twice. Could you please tell me that how did you instantiate the "log" at the beginning ?

Thanks

Saagar said...

@Anonymous

I did instantiation in the normal way.
static Logger log = Logger.getLogger({yourClass}.class);

I am guessing you might have an extra FileAppender in your log4j properties file and as you can see in the code, the comparison is
if(fa instance of FileAppender) it is probably using twice. So you might want to change it to instance of RollingFileAppender or something. I think its probably just something that you overlooked..

Anonymous said...

Hi Saagar,
Thanks for your reply ! We already had changed to RollingFileAppender but not in log4j.properties but in log4j.xml . We have 5 of the RollingFileAppenders as we require to create logs for different services with the hostname , hostip and timestamp appended to the log filename. In the meantime we found a solution to our problem by setting

log.setAdditivity(false);

for each of the appenders for each service.

Whatever your code did help us a lot. Thanks !

Anonymous said...

is this thread safe?

Anonymous said...

I tried it in a WebApp (Struts2) and it looks like it's NOT thread safe. Parallel requests write into the wrong log file here and there. The log4j docs say logger instances are reused if possible - seems to support the suspicion ;-).

Alex Kochnev said...

Beware, this solution only works on a "per run" , where "run" is defined exactly as @Saagar did - one "run" is one JVM execution. The solution above doesn't play well if you have one long running JVM process (as is with typical app servers) - if each thread mucks around w/ the static Log4j config (by executing the sample code in the post), you will end up with issues as described in the comment above - multiple thread's output will get mixed up in the same file (not necessarily the right one).

The solution in the case of a long running process would be based on MDC values , e.g. as described in http://rtner.de/software/MDCUserServletFilter.html

Anonymous said...

Thanks for posting the code. I am using log4j.xml for logging. I am making use of log4j.xml from 6 different Java interfaces. All these java interfaces are placed in UNIX in one common folder.
Now, my problem is that when I run interfaces one after another, all logs are getting captured in one single file that is defined in log4j.xml. But I want interface specific log file. Please let me know from Java end what packages do I have to import to make use of FileAppender methods. And also, what changes should I need to make to log4j.xml to have one log file for one interfaces. Or is there any way to pass the class file name as input parameter to log4j.xml. I have been searching for the solution for a long time but could not find any. Please help me out with this issue.

Anonymous said...

Hi... It is very useful. Thanks for sharing it...

Anonymous said...

thank you very much for your code :D It helped me lot!!!

Anonymous said...

works great... Thanks

V said...

Logger.getRootLogger().getAllAppenders() is always returning null.. But rather I used following code,

code ::

Logger logger=Logger.getLogger(L94Job.class);

Enumeration appenders = logger.getParent().getAllAppenders();

RollingFileAppender fa = null;

while (appenders.hasMoreElements()) {
Appender currAppender = (Appender) appenders.nextElement();
if(currAppender instanceof RollingFileAppender)
{
fa = (RollingFileAppender) currAppender;
}
}
if(fa != null)
{
fa.setFile("home/vaibhav/matson_vmi1.log");
fa.activateOptions();
}

It set value I checked it through debugger. Bt it wont reflect it writes log to _vmi.log file :( :(


Log4j.xml ::

appender name="VMI_FILE" class="org.apache.log4j.RollingFileAppender"
param name="File" value="/home/vaibhav/${file.name}_vmi.log"
param name="MaxFileSize" value="20MB"
param name="MaxBackupIndex" value="20"
param name="Append" value="true"
layout class="org.apache.log4j.PatternLayout"
param name="ConversionPattern"
value="[%d{dd/MM/yy hh:mm:ss:sss z}] %t %5p %c{4}.%M(): %m%n"
layout
appender

I want to dynamically set the filename.Plz help me out I mworking on this issue since long time and project delivery date is just about to come. Rather you can say it passed :|

Thnaks in advanced

atangri said...

i hv a log file created. but at runtiem, now i want to create a new file and get all the contents of previous file in this new file. Do you know how can i achieve this ?

amyt said...

I have a specific requirement to generate specific logs through standalone java application running in linux machine.

Job

Expected Log Location :- {UnixPath} - Dynamic

{UnixPath}/Global.log - contains all the information.
{UnixPath}//NUmber1.log - > Specific Logs only for Number1 processing
{UnixPath}//NUmber2.log Specific Logs only for Number2 processing
{UnixPath}//NUmber3.log Specific Logs only for Number3 processing

Numbers can be 1 to 20 in each order.

I used log4j, I didn't find the logic. Is it possible to achieve this requirement