My WebTrends server is on one side of our firewall and the web servers (with the logs I need to read) are on the other side of the firewall. WebTrends 7 Enterprise doesn't seem to include a secure method for reading the logs or copying them to the WebTrends server for analysis. I can't simply open the appropriate ports on the firewall to map drives or use regular FTP. I can't VPN from the WebTrends server. I'm sure others out there have encountered this problem; how did you solve it?

I'm just curious, what method(s) can you use across the firewall? If there is no way to connect to those servers from across the firewall, then there isn't any magic that WebTrends could use to fix that. Are any ports open at all? How about HTTP(port 80) or HTTPS(443)? We use a script to pull logs for some of our sites via HTTP & HTTPS. Of course, the logs would need to be available from the web server.

Hmmm, using HTTPS is one method the network people didn't mention. How do you do that?

Well, first of all, the logs would need to be available on a web server through HTTPS. I won't cover the details on that.

I use a scheduled task VBScript that takes a list of source addresses and destinations.

Something like this:

https://site.com/logs/server1 \destserver\logs\server1

it's set to try the last X number of days grabbing files

https://site.com/logs/server1/ex050330.log
https://site.com/logs/server1/ex050329.log

and placing them in the destination \destserver\logs\server1\

It's not the prettiest piece of code, but it does the job and is quite reliable. It can send an email in the case of a failure.

Let me know if you're interested and I will clean it up a bit and post it here.

Let me run this by Network and see what they say. Whether thay sign off on it or not, thank you very much.

Hi, me again. My network guys are interested, but say they need to see the entire script before they can make any decision. Would you be willing to post that, or email it to kunklec@hoffman.army.mil? You don't need to clean it up (except to remove your site-identifying info), I don't care if it's messy. Thanks.

Ok, here's the simple version, this one uses an array to define the source and destination paths rather than an external file. I use both, it just depends on your situation. I can provide the other one as well if you want it.

I will email this to you also, as I don't know how it will look on this site. I code in widescreen (long lines), so it might get messed up on these boards.

The biggest problem for you might be that the ID and password are in clear text. I think that there is a way to encrypt VBS files and still run them, but I don't know how secure that really is.

Let me know if you have any questions or problems with this.

' This script downloads log files from the some place over HTTPS
' The "WinHttp.WinHttpRequest.5.1" object is used for the file transfer
' and the "adodb.stream" is used to save the binary response data to a file.
' Unfortunately VBScript doesn't handle binary data with any native data types.
'
'--Begin user variables--

Dim HTTPSUserName = "yourIDhere"
Dim HTTPPassword = "yourpassword here"

Dim EmailAlertDisabled = 1
Dim EmailAlertAddress = "youremail@yourdomain.com"

Dim sSource(3) 'source http path for the file
Dim sDest(3) 'destination path for downloaded file
Dim disabled(3) 'setting to 1 allows disable/skip this path
URLCount = 3

' paths
disabled(0) = 0
sSource(0) = "https://fakesite.com/server1/w3svc1/"
sDest(0) = "\c:\logs\server1\"

disabled(1) = 0
sSource(1) = "https://192.168.1.1/server2/w3svc5/"
sDest(1) = "\servername\logfiles\server2\"

disabled(2) = 1 '<- This one is set to disabled
sSource(2) = "https://othersite/server5/w3svc4/"
sDest(2) = "\servername\d$\logs\server1\w3svc4\"


' HttpRequest SetCredentials flags
HTTPREQUEST_SETCREDENTIALS_FOR_SERVER = 0
HTTPREQUEST_SETCREDENTIALS_FOR_PROXY = 1

' HttpRequest option flags.
' These allow you to ignore any errors that would normally cause a pop-up alert
' dialog in IE. Things like expired cert, etc. In my case, we ignored all errors.
Const Option_SSLErrorIgnoreFlags = 4
Const SslErrorFlag_UnknownCA = 256 ' Unknown certificate authority or untrusted root
Const SslErrorFlag_CertWrongUsage = 512
Const SslErrorFlag_CertCNInvalid = 4096 ' Invalid common name (CN)
Const SslErrorFlag_CertDateInvalid = 8192 ' Invalid date or expired certificate
Const SslErrorFlag_Ignore_All = 13056 ' Ignore all of the above

' ADODB.Stream Constants
Const adTypeBinary = 1
Const adSaveCreateNotExist = 1
Const adSaveCreateOverWrite = 2


' Loop through the various paths/URLs
For x = 0 to (URLCount-1)
if disabled(x) = 1 Then
wscript.echo x & " disabled"
else
sSrcFile = sSource(x) & getLogFileName(-1)
wscript.echo sSrcFile
sDestFile = sDest(x) & getLogFileName(-1)
wscript.echo sDestFile
DownloadFile sSrcFile, sDestFile
End If

Next

function DownloadFile(URLString, sDestination)
on error resume next
Set oWinHttpReq = CreateObject ("WinHttp.WinHttpRequest.5.1")
Set oStream = createobject("adodb.stream")
oWinHttpReq.Open "GET", URLString, false
oWinHttpReq.SetCredentials HTTPSUserName, HTTPSPassword, HTTPREQUEST_SETCREDENTIALS_FOR_SERVER
oWinHttpReq.Option(4) = 13056
oWinHttpReq.Send
If (Err <> 0) then
MsgBodyText = "There was an error downloading " & URLString & ". Connection to server could not be established."
SendMail MsgBodyText, EmailAlertAddress, "HTTPS Script Failure"
else
wscript.echo oWinHttpReq.Status
If oWinHttpReq.Status = 200 Then
oStream.type = adTypeBinary
oStream.open
oStream.write oWinHttpReq.ResponseBody
oStream.savetofile sDestination, adSaveCreateOverWrite
Else
MsgBodyText = "There was an error downloading " & URLString & ". The request returned HTTP code " & oWinHttpReq.Status & "."
SendMail MsgBodyText, EmailAlertAddress, "HTTPS Script Failure"
End If
set oStream = nothing
set oWinHttpReq = nothing
end if
end function

' Returns a string of format exYYMMDD.zip dayshift allows both
' pos/neg values for the day shift zero for today
function getLogFileName(dayshift)
theDate = DateAdd("d",dayshift,now) ' subtract a day from now

d = datepart("d",theDate) 'get the day
d = string(2-len(d),"0") & d 'pad with zero

m = datepart("m",theDate) 'get the month
m = string(2-len(m),"0") & m 'pad with zero

y = datepart("yyyy",theDate) 'get year
y = right(y,2) 'take only last two digits

getLogFileName = "ex" & y & m & d & ".zip"
end function

Sub SendMail(text, address, subject)
If Not (EmailAlertDisabled = 1) Then
Dim objNewMail
Set objNewMail = CreateObject("CDONTS.NewMail")
objNewMail.From = "webtrends@ford.com"
objNewMail.To = address
objNewMail.Subject = subject
objNewMail.Body = text
objNewMail.Send
Set objNewMail = Nothing
End If
End Sub