+ Reply to Thread
Results 1 to 14 of 14

Mass data import from 200 files... automatically?

  1. #1
    Forum Contributor
    Join Date
    03-24-2009
    Location
    Vietnam
    MS-Off Ver
    Excel 2010
    Posts
    382

    Mass data import from 200 files... automatically?

    Hello!

    I think this time a have a special little problem to figure out for our excel lords in here ^^

    I have around 200 files, all named from 001.xls to 200.xls, all containing the same format, the same number of colums, but a different number of lines.

    Now I have the great task to actually open all these files and copy each files data into one file and start analyzing it. Well, since I personally have no interest to open, copy and paste 200 hundred times, I am wondering if there is any faster way to let excel automatically copy the data into this master file?
    I am thinking about somehing like *='001.xls'Sheet2!B2:E20*

    One thing is that each file never has more than 20 entries, thus i would only need excel to import the from line 2 to 20 from a certain file and then do same thing all over again with the next file.

    Every suggestion is welcomed!
    Thanks a lot!
    A2k
    Last edited by Armitage2k; 04-24-2009 at 03:55 PM.

  2. #2
    Registered User
    Join Date
    12-23-2007
    Posts
    47

    Re: Mass data import from 200 files... automatically?

    If the files are always stored in the same location with the same file names then you could just create the string for the file and then open the file.

    strfile="D:\desktop\" & filecount & ".xls"
    open file strfile

    If the files are diffent each time, then this is a way that you can select the files you want to work with.

    You select all files that are to be opened, and then you go through the files one by one opening them, and taking the information you need.


    [code]
    'Open file selection menu
    FName = Application.GetOpenFilename("Excel Files,*.xls", 1, "Select all files", , True)
    If TypeName(FName) = "Boolean" Then
    MsgBox "You didn't select a file"
    Exit Sub
    End If

    'Moves through array of file name until all have been processed
    For f = 1 To UBound(FName)
    Debug.Print "Selected file #" & f & ": " & FName(f)
    Application.ScreenUpdating = False
    Workbooks.OpenText FileName:=FName(f)
    Call Harvest
    Next f
    ActiveWindow.Close
    [\code]

  3. #3
    Forum Contributor
    Join Date
    03-24-2009
    Location
    Vietnam
    MS-Off Ver
    Excel 2010
    Posts
    382

    Re: Mass data import from 200 files... automatically?

    well, would be boring if it is that easy...

    I guess this is the right time to mention that I am not really familiar with VBA and just use it too rarely. however, i went with the first solution and am stuck with a syntax error I cannot figure out. Please see my attached example for closer details.

    about the problem itself:

    I _think_ you missunderstood my problem. I DONT want to have to open every single file and manually copy the date i need, I want excel to do this. this means, every file, from A1 (or A2, first line are the headlines) to at least K20 (10 columns) want should be copied into this one master file, on Sheet1. The result should be that I have all the data from all files in my master file.

    And yes, all the files are in the same folder and all are named 001.xls, 002.xls, etc. .
    thanks for the help!
    A2k
    Attached Files Attached Files
    Last edited by Armitage2k; 04-24-2009 at 10:08 AM.

  4. #4
    Registered User
    Join Date
    12-23-2007
    Posts
    47

    Re: UNSOLVED - Mass data import from 200 files... automatically?

    Here is the jist of the code.

    Here is what you want to do:

    You want to count through all the file number so you have the loop.
    Next you construct the string that tells where to go for the files
    Select the range of cells
    close workbook
    Paste selection into your new file
    Repeat

    PHP Code: 

    for 1 to 200 
    strFile
    ="C:\test\data\" & x & ".xls"
    openfile strfile

    range("
    A3:R25").select
    Selection.copy
    activeworkbook.close

    Cells(x*2,1).paste 

    next x 

  5. #5
    Forum Contributor
    Join Date
    03-24-2009
    Location
    Vietnam
    MS-Off Ver
    Excel 2010
    Posts
    382

    Re: UNSOLVED - Mass data import from 200 files... automatically?

    OK thanks, the code is clear, but I get a compiling error everytime I try to run the makro.
    Heres what I do:

    Open Master file, enter VBA and create makro for Sheet1.
    PHP Code: 
    Sub MassImport()
    For 
    1 To 200
    strfile 
    "C:\test\files\" & x & ".xls"
    openfile strfile

    Range("
    A3:R25").Select
    Selection.Copy
    ActiveWorkbook.Close

    Cells(x * 2, 1).Paste

    Next x
    End Sub 
    When I try to run it, I always get the error message: Error during compiling and the highlighted area points to "Sub Massimport()"

    thanks for the help!
    A2k

  6. #6
    Registered User
    Join Date
    12-23-2007
    Posts
    47

    Re: UNSOLVED - Mass data import from 200 files... automatically?

    PHP Code: 
    Sub MassImport()
    For 
    1 To 200
    strfile 
    "C:\test\files\" & x & ".xls"
    Workbooks.OpenText strfile

    Range("
    A3:R25").Select
    Selection.Copy
    ActiveWorkbook.Close

    Cells(x * 2, 1).Paste

    Next x
    End Sub 

  7. #7
    Forum Contributor
    Join Date
    03-24-2009
    Location
    Vietnam
    MS-Off Ver
    Excel 2010
    Posts
    382

    Re: UNSOLVED - Mass data import from 200 files... automatically?

    sorry for being such a pest, but there is still some debugging error:

    whenever i run the macro i get the message that there is a lot of data in the clipboard and if i would like to keep it for later use in other programs.
    if i press yes, i get the error action is not supported, and when pressing debugging, i get the following area highlighted.
    PHP Code: 
    Cells(21).Paste 
    Generally, i need a way how the specified area is copy automatically into my masterfile, and then continues with the next file, preferably without any notice at all. its always like open one file, choose the first 20 lines, copy to master file. open next file, choose 20 first lines, copy to master file (after the first entry of course. Same thing all over until all files have been cleared.

    Again, sorry for be so annoying, but i really have no clue about vba yet.
    thanks!
    A2k

  8. #8
    Forum Contributor
    Join Date
    10-30-2007
    Location
    Norway
    MS-Off Ver
    MS Office 2007
    Posts
    345

    Re: UNSOLVED - Mass data import from 200 files... automatically?

    I have made some modifications to your code, so it should work as you requested.

    You have to put this i a Module and not in the code for sheet1. In the VBA editor you chose Insert->Module. You should then get a Module1 where you can put this code.
    Sub MassImport()
    Dim FileName As String
    Dim strfile As String
    Dim x As Integer
    Dim Source As Excel.Workbook

    For x = 1 To 200
    If x < 10 Then
    FileName = "00" & x
    ElseIf x < 100 Then
    FileName = "0" & x
    Else
    FileName = x
    End If

    strfile = "C:\test\files\" & FileName & ".xls"
    Set Source = Workbooks.Open(strfile)
    Source.Sheets(1).Range("A3:R25").Copy Destination:=ThisWorkbook.Sheets(1).Cells(x * 23, 1)
    Source.Close

    Next x
    End Sub

  9. #9
    Registered User
    Join Date
    12-23-2007
    Posts
    47

    Re: Mass data import from 200 files... automatically?

    Add this code before you start doing anything and then add again after and change the setting to true.

    This will supress the screen updates while the code is running.

    PHP Code: 
        Application.ScreenUpdating False
        Application
    .DisplayAlerts False 

  10. #10
    Forum Contributor
    Join Date
    01-13-2009
    Location
    Mumbai
    MS-Off Ver
    Excel 2003, 2007
    Posts
    168

    Re: Mass data import from 200 files... automatically?

    At the end of the code you should set the properties to default..

    Please Login or Register  to view this content.

  11. #11
    Forum Expert martindwilson's Avatar
    Join Date
    06-23-2007
    Location
    London,England
    MS-Off Ver
    office 97 ,2007
    Posts
    19,320

    Re: Mass data import from 200 files... automatically?

    here's one i use
    put in same directory as your 200+files you want to extract info from (no otherunrelated xl files should be in there )
    view code change range and increment row count where indicated (its currently set for a2:k20 with increments of 19 rows)
    type name of sheet into c7 (i assume all use the same sheet name,if they dont this wont work)
    click init
    click then click treat file
    everything should now be copied to sheet data.
    Attached Files Attached Files
    "Unless otherwise stated all my comments are directed at OP"

    Mojito connoisseur and now happily retired
    where does code go ?
    look here
    how to insert code

    how to enter array formula

    why use -- in sumproduct
    recommended reading
    wiki Mojito

    how to say no convincingly

    most important thing you need
    Martin Wilson: SPV
    and RSMBC

  12. #12
    Forum Contributor
    Join Date
    03-24-2009
    Location
    Vietnam
    MS-Off Ver
    Excel 2010
    Posts
    382

    Re: Mass data import from 200 files... automatically?

    wonderful, thanks for the great support. I tried both versions and they equally work absolutely fine. just one quick question for Martin D. Wilson and his solution:

    Can you define from which cell onwards I want the script to start pasting? Default is A1, but I would like to add some headlines, thus would need a paste starting from A2, but if I change the parking cell in the script, nothing happens...

    OK, I have one more short question for you people.
    After I have imported all of my files, i figured i have a few entries twice or three times. But since I am talking about 7000 lines, i do not want to start looking through every entry and check if it only exists once, respectively dont want to sort all the lines and see any double entries.

    Is there a way a macro can check if the contents from (eg) A1:K1 exist again in that exact order somewhere between A1:K7000 ?
    the good thing is that the entry would be EXACTLY the same, but I still have a few which are a just a little bit different and those must be kept.

    Again, thanks for all the great support!
    A2k

  13. #13
    Forum Expert martindwilson's Avatar
    Join Date
    06-23-2007
    Location
    London,England
    MS-Off Ver
    office 97 ,2007
    Posts
    19,320

    Re: Mass data import from 200 files... automatically?

    Can you define from which cell onwards I want the script to start pasting? Default is A1, but I would like to add some headlines, thus would need a paste starting from A2, but if I change the parking cell in the script, nothing happens...
    err add the headers afterwards
    you could change
    this bit
    Please Login or Register  to view this content.
    to
    Please Login or Register  to view this content.
    Last edited by martindwilson; 04-27-2009 at 12:12 PM.

  14. #14
    Forum Contributor
    Join Date
    10-30-2007
    Location
    Norway
    MS-Off Ver
    MS Office 2007
    Posts
    345

    Re: Mass data import from 200 files... automatically?

    To remove the duplicates you can use Advanced Filter. You Find it on the Data-Filter menu.

    Select : "Copy to another location" and "Unique Records only".
    "List Range", is the data you want to filter and "Copy To" is where you want to paste the result (probably another sheet). Leave "Criteria range" empty.

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1