Shell script for complex file manipulation

March 29, 2012 at 06:19:21
Specs: CentOS
I'm writing a shell script to retrieve some data from a text file and I'm really confused over how to sort out one particular issue.

I have a text file which is something like this.

# Some random data #

Current login id abc123
# List of requests #

Current login id aa111
# List of requests #

Current login id bbb222
# List of requests #

Current login id ccc333
# List of requests #

Current login id ddd444
# List of requests #

# Some random data #


Each request is of the foll form

DB name : database1
server name : serv1
access type : read_only_access


There and multiple login ids and each login id has multiple requests.

I'm trying to separate each login id and it's set of requests and put them in a separate text file and ignore the random data.

That is, I want to start with the first login id and copy all the requests following it and put them in a separate text file from where i will be running other operations on the text.

When i have completed all the operations with the text, i want the next login id and the requests following it to be copied to the same text file(overwritten), so that i can run the same operations on them, and so on.

I'm unable to come up with the right code to implement this operation.

Please help.


See More: Shell script for complex file manipulation

Report •

#1
March 29, 2012 at 06:29:55
I want my script to perform the following process.


# Start searching for the line starting with "Current login id"
# Ignore or delete all random data till the line "Current login id" is found
# Once the line is found, copy that line and all the lines following it to another text file,
# Keep copying all the lines until the next line starting with "Current login id" is encountered.
# Stop copying at that point, and delete all the copied lines from the original text file.
# Perform some operations on the copied data and then erase the copied file.
# Continue this operation till the original text file is empty.


This is the logic I've come up with, but I'm still unable to write the exact code for this.

Please help.


Report •

#2
March 29, 2012 at 14:50:51
How about using the bash shell's regular expression comparison operator =~
I also place the 3 requests into the file and forget about everything else. When the while loop terminates, the data for one login still exists in the file:

#!/bin/bash

> sepfile.txt
while read line
do
   # login changes, perform your operations if the
   # sepfile.txt file size is greater than zero
   if [[ $line =~ "^Current login id" ]]
   then
      if [[ -s sepfile.txt ]]
      then
          echo "perform some operations"
      fi
      # recreate the file
      echo "$line" > sepfile.txt
   fi
   # capture the requests
   if [[ ($line =~ "^DB name") || ($line =~ "^server name") || ($line =~ "^access type") ]]
   then
      echo "$line" >> sepfile.txt
   fi
done < datafile.txt
# pick up the last login
if [[ -s sepfile.txt ]]
then
    echo "perform some operations"
fi


Report •

#3
March 30, 2012 at 03:04:53
Nails,

Your code works really fine, but I'm still unable to understand parts of it.

If i want to print the result of each iteration onto a new text file, where do i add the code for that ?

i.e., when the first iteration is completed, sepfile.txt will contain the first login id and it's requests. I want to print this data to a text file called file1.txt

after the second iteration, sepfile.txt will contain the second login id and it's requests. I want to print this data to a text file file2.txt

I will use a variable "a" and keep incrementing it's value and specify the file name as file$a.txt.

The lines of code i'll be using are

cat sepfile >> file$a.txt
a=`expr $a + 1`


Please tell me where exactly in your code I should add these lines to achieve my goal ?


Report •

Related Solutions

#4
March 30, 2012 at 03:43:30
Nails,

It just hit me. Now i understand it all.

It's so complex, yet so simple.

I added these two lines

cat sepval.txt > filename$a.txt
a=`expr $a + 1`

after the 11th line ( The first echo "perform some operations" line).

Now i get all the requests of each login id in separate text files and the requests of the last login id in sepfile.txt.

Thanks a lot for your help !


Report •

#5
April 19, 2012 at 05:40:50
Nails,

SOS.
I've encountered a new error.
There are a few more lines in my text file that start with the words "Server name", but they are unnecessary.

The lines that hold the real server names start something like Server (server_name).

Now, I have no idea of predicting how the server name will be. So i have to search for it using the pattern starting with "Server (".

I have to implenemt this in line no 18 of your code given above.

How do i implement Server (* in this place?

* stands for some unknown text following the pattern Server (


Report •

#6
April 19, 2012 at 09:04:00
I am interpreting that if the line starts with the string "Server(", parse the server name that is within the parenthesis?

#!/bin/bash

line="Server(Sanjeev_server) more text"
if [[ ($line =~ "^Server\(") ]]
then
   server_name=$(echo "$line" | cut -d'(' -f2 | cut -d')' -f1)
   echo $server_name  # should display Sanjeev_server
fi


Report •

#7
April 25, 2012 at 00:51:20
Nails,

Thanks a lot. This is exactly what i was looking for. It works flawlessly.

I was trying out a code similar to this using ~ "^Server\(*)

instead of ~ "^Server\(")

Maybe that's why i was unable to get the derisred results.

Thanks again for helping me out.


Report •

Ask Question