Battlelog Cache (1.0.1.0 - 14/1/13)

[ImportBot]

Originally Posted by PapaCharlie9*:

 

Battlelog Cache

---

Author: MorpheusX(AUT)

 

BATTLELOG CACHE DOES NOT WORK FOR BF4 AND MOST LIKELY NEVER WILL.

Description

---

This plugin should be used by plugin-authors to query for Battlelog-stats. It uses a MySQL-Database to cache requests and thus allows reducing the load on Battlelog.

 

This version if the very first public release and thus may contain bugs, however it has proven stable over a few days of testing on multiple servers. If you encounter any bugs, please inform the plugin author.

 

Whilst this caching-plugin will reduce the amount of requests made towards Battlelog, it doesn't solve the 'Too Many Requests' problem completely. The first version does not contain dynamic throtteling and re-fetching of failed requests yet.

 

Commands

---

Battlelog Cache does not include any ingame-commands or additional functionality, but just provides an interface for other plugins to query. In order to communicate with Battlelog Cache, a plugin must implement the following mechanisms:

• Provide a public method following the given scheme: public void METHODNAME(params String[] response) { }
• Check whether Battlelog Cache is installed by issueing this.GetRegisteredCommands(); and parsing the list for a MatchCommand with RegisteredClassname == CBattlelogCache and RegisteredMethodName == PlayerLookup
• Create a Hashtable with the following keys: playerName, pluginName, pluginMethod, requestType
• Execute a query calling the following command: this.ExecuteCommand("procon.protected.plugins.call ", "CBattlelogCache", "PlayerLookup", JSON.JsonEncode(HASHTABLE));

• METHODNAME would be the name of the plugin's public method for returning stats
• playerName should have the name of the player as a content
• pluginName should have the Class-Name of a plugin as a content (e.g. CBattlelogCache)
• pluginMethod should have METHODNAME as a content
• requestType defines the type of the statsrequest. Possible requests: 'overview', 'weapon', 'vehicle', 'clanTag'
• HASHTABE should be replaced with the hashtable created in the step before

Settings

---

Debug level

Indicates how much debug-output is printed to the plugin-console. 0 turns off debug messages (just shows important warnings/exceptions), 6 documents nearly every step.

MySQL Hostname

Hostname of the MySQL-Server Battlelog Cache should connect to.

MySQL Port

Port of the MySQL-Server Battlelog Cache should connect to. (Default: 3306)

MySQL Database

Database Battlelog Cache should use to cache stats.

MySQL Table

Table Battlelog Cache should use to cache stats. (Default: playerstats)

MySQL Username

Username Battlelog Cache should use to authenticate with the MySQL-Server.

MySQL Password

Password Battlelog Cache should use to authenticate with the MySQL-Server.

If you don't see the MySQL settings, the MySQL details have been 'hardcoded' in the Configs/gsp_settings/battlelogcache_mysql.cfg file. Hosters can use this to 'enforce' all users to query a centralized database, allowing more cache-hits.

 

To disable the MySQL settings, create a subfolder gsp_settings within the Configs folder in Procon. Within this subfolder, create a file called battlelogcache_mysql.cfg. This file should have the following structure:

Code:

hostname HOSTNAME
port PORT
database DATABASE
table TABLE
username USERNAME
password PASSWORD

MySQL Setup

---

In order for Battlelog Cache to work, it needs to be connected to a MySQL-Database. You can either get one from a hosting provider or you can set one up yourself, as long as you have network connectivity through your firewall to your Procon instances (instructions for setting up a MySQL database server are here: [HowTo] Set up a MySQL server for Procon*

 

The needed CREATE-script for the table can either be found in the plugin's zip-File or manually downloaded at this page.

 

NOTE: if you use the script as-is, you must first do a

Code:

use [i]your_db_name[/i];

command, where your_db_name is the name of the database you granted access to in the [HowTo] Set up a MySQL server for Procon instructions. Alternatively, you can customize the script by replacing the first line of the script with this:

Code:

CREATE TABLE IF NOT EXISTS `[i]your_db_name[/i]`.`playerstats` (

You should of course replace the your_db_name with the actual name you picked.

 

After the table has been created, no further setup will be required to run Battlelog Cache.

 

Download

---

This plugin is bundled with Procon as of version 1.4.0.5. There is no need to download or install it, it should already be installed.

 

Changelog

1.0.1.0 (14.1.2013)

- added more error handling, reduced spamming of plugin.log for setting 2 or lower

1.0.0.0 (18.12.2012)

- initial (public) version

Attached Files:

• CBattlelogCache.zip

[ImportBot]

Originally Posted by PapaCharlie9*:

 

Don't get all excited, I'm just giving Morpheus a hand and posting this for him, so you guys have one place to track support issues and updates.

[ImportBot]

Originally Posted by LumpyNutZ*:

 

Sorry this stupid question but, from where to get that 1.0.0.1?

[ImportBot]

Originally Posted by MorpheusX(AUT)*:

 

It's not released yet. PC9 made some last changes yesterday evening and sent them to me for a last check. Since Phil is on holiday, I'm not sure I will succeed in adding it to the Package System, but we'll attach it to this post during the day.

[ImportBot]

Originally Posted by LumpyNutZ*:

 

It's not released yet. PC9 made some last changes yesterday evening and sent them to me for a last check. Since Phil is on holiday, I'm not sure I will succeed in adding it to the Package System, but we'll attach it to this post during the day.

Ok thanks for clearing this.

[ImportBot]

Originally Posted by jstm*:

 

How various statistic will be aged for already cached users ? I mean how will be decided that some certain stats should be refreshed against battlelog ?

[ImportBot]

Originally Posted by EBassie*:

 

How various statistic will be aged for already cached users ? I mean how will be decided that some certain stats should be refreshed against battlelog ?

They will be refreshed after 24 hours, if I'm correct.

[ImportBot]

Originally Posted by LumpyNutZ*:

 

@PC9

That logs, how often u wanna get em?

[ImportBot]

Originally Posted by PapaCharlie9*:

 

@PC9

That logs, how often u wanna get em?

Just once and only if you have had a HTTP request error, like Too Many Requests or Timed Out. I just need to confirm that HTTP request errors are handled as expected, which is, you get some kind of message (just one!), no NullReferenceExceptions, and nothing gets entered in your database for that player, not even an error. The idea is that an HTTP request error is temporary and the next request might work, so we don't want to throw a 24 hour delay in if it is not necessary.

[ImportBot]

Originally Posted by LumpyNutZ*:

 

PC9 just sent the logs for you, let me know if you need more.

[ImportBot]

Originally Posted by Hutchew*:

 

Any ideas what is causing this? battlelog cache enabled using same MYSQL Settings as BF3 Chat, GUID, stats and Map Logger plugin (which is not generating errors). playerstats table has been created successfully. Throws this error when enabling Insane Limits plugin:

 

[17:06:25 25] [battlelog Cache] [battlelog Cache] EXCEPTION: System.OverflowException: Arithmetic operation resulted in an overflow.

at MySql.Data.MySqlClient.Authentication.MySqlAuthent icationPlugin.HandleAuthChange(MySqlPacket packet)

at MySql.Data.MySqlClient.Authentication.MySqlAuthent icationPlugin.Authenticate(Boolean reset)

at MySql.Data.MySqlClient.NativeDriver.Authenticate(S tring authMethod, Boolean reset)

at MySql.Data.MySqlClient.NativeDriver.Open()

at MySql.Data.MySqlClient.Driver.Open()

at MySql.Data.MySqlClient.Driver.Create(MySqlConnecti onStringBuilder settings)

at MySql.Data.MySqlClient.MySqlPool.CreateNewPooledCo nnection()

at MySql.Data.MySqlClient.MySqlPool.GetPooledConnecti on()

at MySql.Data.MySqlClient.MySqlPool.TryToGetDriver()

at MySql.Data.MySqlClient.MySqlPool.GetConnection()

at MySql.Data.MySqlClient.MySqlConnection.Open()

at PRoConEvents.CBattlelogCache.MySqlInsert(Object insertData)

[ImportBot]

Originally Posted by ty_ger07*:

 

Any ideas what is causing this? battlelog cache enabled using same MYSQL Settings as BF3 Chat, GUID, stats and Map Logger plugin (which is not generating errors). playerstats table has been created successfully. Throws this error when enabling Insane Limits plugin:

 

[17:06:25 25] [battlelog Cache] [battlelog Cache] EXCEPTION: System.OverflowException: Arithmetic operation resulted in an overflow.

at MySql.Data.MySqlClient.Authentication.MySqlAuthent icationPlugin.HandleAuthChange(MySqlPacket packet)

at MySql.Data.MySqlClient.Authentication.MySqlAuthent icationPlugin.Authenticate(Boolean reset)

at MySql.Data.MySqlClient.NativeDriver.Authenticate(S tring authMethod, Boolean reset)

at MySql.Data.MySqlClient.NativeDriver.Open()

at MySql.Data.MySqlClient.Driver.Open()

at MySql.Data.MySqlClient.Driver.Create(MySqlConnecti onStringBuilder settings)

at MySql.Data.MySqlClient.MySqlPool.CreateNewPooledCo nnection()

at MySql.Data.MySqlClient.MySqlPool.GetPooledConnecti on()

at MySql.Data.MySqlClient.MySqlPool.TryToGetDriver()

at MySql.Data.MySqlClient.MySqlPool.GetConnection()

at MySql.Data.MySqlClient.MySqlConnection.Open()

at PRoConEvents.CBattlelogCache.MySqlInsert(Object insertData)

Check that the server hosting your MySQL database is using new password encryption of MySQL passwords. If it is using old password encryption, the .NET MySQL connector gives error "Arithmetic operation resulted in an overflow" because old password encryption has become deprecated.

[ImportBot]

Originally Posted by Hutchew*:

 

Thanks, ty_ger07. Closely related; a password reset did the trick.

[ImportBot]

Originally Posted by Hutchew*:

 

Deleted: problem already posted in Procon General.

[ImportBot]

Originally Posted by Legate*:

 

Houston I have a problem:

 

Code:

[18:53:02 25] [Battlelog Cache] EXCEPTION: Exception in RequestLoop!
[18:53:02 35] [Battlelog Cache] EXCEPTION: System.NullReferenceException: Object reference not set to an instance of an object.
   at PRoConEvents.CBattlelogCache.BattlelogLookup(PlayerStats stats, RequestTypeEnum requestType)
   at PRoConEvents.CBattlelogCache.PerformLookup(String name, RequestTypeEnum requestType, DateTime startTime, Double minWaitTime)
   at PRoConEvents.CBattlelogCache.RequestLoop()

The table was well created (It has 7 rows at this time) but I don't know what could be the error that generate the message....

 

Any suggestion?

 

Note: I'm using the 1.0.0.0 version

[ImportBot]

Originally Posted by PapaCharlie9*:

 

Houston I have a problem:

 

Code:

[18:53:02 25] [Battlelog Cache] EXCEPTION: Exception in RequestLoop!
[18:53:02 35] [Battlelog Cache] EXCEPTION: System.NullReferenceException: Object reference not set to an instance of an object.
   at PRoConEvents.CBattlelogCache.BattlelogLookup(PlayerStats stats, RequestTypeEnum requestType)
   at PRoConEvents.CBattlelogCache.PerformLookup(String name, RequestTypeEnum requestType, DateTime startTime, Double minWaitTime)
   at PRoConEvents.CBattlelogCache.RequestLoop()

The table was well created (It has 7 rows at this time) but I don't know what could be the error that generate the message....

 

Any suggestion?

 

Note: I'm using the 1.0.0.0 version

We're testing a patch, should be available in a few days.

[ImportBot]

Originally Posted by Legate*:

 

Ah excellent, do you need any log or something like that?

[ImportBot]

Originally Posted by PapaCharlie9*:

 

As an expedient, the 1.0.1.0 patch has been attached to post #1 above. At some time in the future it will be included in the next update for Procon, but if you would rather not wait, you can download it from post #1 and install it now.

 

The patch removes the NullReferenceException for stats fetch errors. It still logs messages when stats fetching fails, but the messages are shorter/fewer lines at Debug level of 2. If you don't want to see status messages about failed stats fetches at all, set Debug level to 1.

 

Even if you lower your logging level, you can still recover detailed error information about stats fetching from your cache, if you have SQL query access. Execute the following SQL query to dump all player records that had an error in the last cache refresh:

 

Code:

SELECT personaId, clanTag, playerName, generalStatus, overviewStatsError, weaponStatsError 
FROM [b]mydb[/b].playerstats
WHERE generalStatus <> 'Success';

Replace mydb with the name of your database. If you changed the name of the database table in your configuration, change playerstats as well.

 

Many thanks to EBastard and LumpyNutz for their help in testing this patch.

[ImportBot]

Originally Posted by aduh*:

 

[battlelog Cache] MySql.Data.MySqlClient.MySqlException: Host '216.38.9.13' is not allowed to connect to this MySQL server

I tried with many host sites and can't connect to the MYSQL server.

Any suggestions with free mysql hosting that will work for sure ?

[ImportBot]

Originally Posted by Pallywhacker*:

 

I have set up my MySQL server (Great instructions by the way Papa). Now my question is, does the Battlelog Cache 1.0.1.0 version automaticly create the table? I saw the link to this code,

 

CREATE TABLE IF NOT EXISTS `playerstats` (

`personaId` varchar(50) COLLATE utf8_bin NOT NULL,

`playerName` varchar(32) COLLATE utf8_bin NOT NULL,

`clanTag` varchar(8) COLLATE utf8_bin DEFAULT NULL,

`overviewStats` longtext COLLATE utf8_bin,

`weaponStats` longtext COLLATE utf8_bin,

`vehicleStats` longtext COLLATE utf8_bin,

`generalStatus` text COLLATE utf8_bin,

`overviewStatsError` text COLLATE utf8_bin,

`weaponStatsError` text COLLATE utf8_bin,

`vehicleStatsError` text COLLATE utf8_bin,

`timestampOverview` timestamp NULL DEFAULT NULL,

`timestampWeapon` timestamp NULL DEFAULT NULL,

`timestampVehicle` timestamp NULL DEFAULT NULL,

`timestampClanTag` timestamp NULL DEFAULT NULL,

PRIMARY KEY (`personaId`),

UNIQUE KEY `playerName` (`playerName`)

) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

 

I ran it and received an error. I'm not sure that I did properly. How do I verify that the table was setup?

 

Thank you in advance.

[ImportBot]

Originally Posted by XpKiller*:

 

What was the error message?

[ImportBot]

Originally Posted by Pallywhacker*:

 

The error was no database selected. Apparently I had not made or selected a database. Once I made a database and selected it, I ran the code to create the table and all went fine. Xpkiller thank you for your quick reponse though.

[ImportBot]

Originally Posted by PapaCharlie9*:

 

The error was no database selected. Apparently I had not made or selected a database. Once I made a database and selected it, I ran the code to create the table and all went fine. Xpkiller thank you for your quick reponse though.

The last section of the [HowTo] instructions has you create a database. Did you miss that step? Or were you not sure whether you needed to do it or not? I ask because if a clarification is needed in the [HowTo], I want to add it.

[ImportBot]

Originally Posted by maester*:

 

Im not sure what this error means but I receive this when the plugin is enabled

 

EXCEPTION: MySql.Data.MySqlClient.MySqlException: Authentication method 'mysql_old_password' not supported by any of the available plugins.

at MySql.Data.MySqlClient.Authentication.Authenticati onPluginManager.GetPlugin(String method)

at MySql.Data.MySqlClient.Authentication.MySqlAuthent icationPlugin.GetPlugin(String method, NativeDriver driver, Byte[] authData)

at MySql.Data.MySqlClient.Authentication.MySqlAuthent icationPlugin.HandleAuthChange(MySqlPacket packet)

at MySql.Data.MySqlClient.Authentication.MySqlAuthent icationPlugin.Authenticate(Boolean reset)

at MySql.Data.MySqlClient.NativeDriver.Authenticate(S tring authMethod, Boolean reset)

at MySql.Data.MySqlClient.NativeDriver.Open()

at MySql.Data.MySqlClient.Driver.Open()

at MySql.Data.MySqlClient.Driver.Create(MySqlConnecti onStringBuilder settings)

at MySql.Data.MySqlClient.MySqlPool.CreateNewPooledCo nnection()

at MySql.Data.MySqlClient.MySqlPool.GetPooledConnecti on()

at MySql.Data.MySqlClient.MySqlPool.TryToGetDriver()

at MySql.Data.MySqlClient.MySqlPool.GetConnection()

at MySql.Data.MySqlClient.MySqlConnection.Open()

at PRoConEvents.CBattlelogCache.MySqlInsert(Object insertData)

[ImportBot]

Originally Posted by Pallywhacker*:

 

The last section of the [HowTo] instructions has you create a database. Did you miss that step? Or were you not sure whether you needed to do it or not? I ask because if a clarification is needed in the [HowTo], I want to add it.

Papa,

 

Sorry for the late reply. I just noticed your question today. I just missed the setup data base part of your instructions. For someone like myself who had no clue to what I was doing with the MYSQL setup. It was somewhat over whelming at times during the process. So by the time I got to the end of the setup I was tired and missed it. The one thing I would add to the instructions is that after you use the

 

Code:

Show Databases;

command

 

You should tell them to use the

 

Code:

use db name;

command

 

to select the database to use so that when you use this code

 

Code:

CREATE TABLE IF NOT EXISTS `playerstats` (
`personaId` varchar(50) COLLATE utf8_bin NOT NULL,
`playerName` varchar(32) COLLATE utf8_bin NOT NULL,
`clanTag` varchar(8) COLLATE utf8_bin DEFAULT NULL,
`overviewStats` longtext COLLATE utf8_bin,
`weaponStats` longtext COLLATE utf8_bin,
`vehicleStats` longtext COLLATE utf8_bin,
`generalStatus` text COLLATE utf8_bin,
`overviewStatsError` text COLLATE utf8_bin,
`weaponStatsError` text COLLATE utf8_bin,
`vehicleStatsError` text COLLATE utf8_bin,
`timestampOverview` timestamp NULL DEFAULT NULL,
`timestampWeapon` timestamp NULL DEFAULT NULL,
`timestampVehicle` timestamp NULL DEFAULT NULL,
`timestampClanTag` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`personaId`),
UNIQUE KEY `playerName` (`playerName`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

The table is setup in their database for battlelog cache. I hope that my response is what you were looking for.

[ImportBot]

Originally Posted by PapaCharlie9*:

 

You should tell them to use the

 

Code:

use db name;

command

 

to select the database to use so that when you use this code

 

Code:

CREATE TABLE IF NOT EXISTS `playerstats` (
`personaId` varchar(50) COLLATE utf8_bin NOT NULL,
`playerName` varchar(32) COLLATE utf8_bin NOT NULL,
`clanTag` varchar(8) COLLATE utf8_bin DEFAULT NULL,
`overviewStats` longtext COLLATE utf8_bin,
`weaponStats` longtext COLLATE utf8_bin,
`vehicleStats` longtext COLLATE utf8_bin,
`generalStatus` text COLLATE utf8_bin,
`overviewStatsError` text COLLATE utf8_bin,
`weaponStatsError` text COLLATE utf8_bin,
`vehicleStatsError` text COLLATE utf8_bin,
`timestampOverview` timestamp NULL DEFAULT NULL,
`timestampWeapon` timestamp NULL DEFAULT NULL,
`timestampVehicle` timestamp NULL DEFAULT NULL,
`timestampClanTag` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`personaId`),
UNIQUE KEY `playerName` (`playerName`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

The table is setup in their database for battlelog cache. I hope that my response is what you were looking for.

That's a good tip, thanks!

 

An alternative is to change the create to use `mydb`.`playerstats` (or whatever your db name is for mydb).

[ImportBot]

Originally Posted by PapaCharlie9*:

 

Thanks to feedback from Pallywhacker, updated the instructions to include defining the database before the create table command.

[ImportBot]

Originally Posted by HexaCanon*:

 

[20:23:04 96] [battlelog Cache] MySql.Data.MySqlClient.MySqlException: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding. ---> System.TimeoutException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond ---> System.IO.IOException: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)

at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)

--- End of inner exception stack trace ---

at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)

at MySql.Data.Common.MyNetworkStream.Read(Byte[] buffer, Int32 offset, Int32 count)

--- End of inner exception stack trace ---

at MySql.Data.Common.MyNetworkStream.HandleOrRethrowE xception(Exception e)

at MySql.Data.Common.MyNetworkStream.Read(Byte[] buffer, Int32 offset, Int32 count)

at MySql.Data.MySqlClient.TimedStream.Read(Byte[] buffer, Int32 offset, Int32 count)

at System.IO.BufferedStream.Read(Byte[] array, Int32 offset, Int32 count)

at MySql.Data.MySqlClient.MySqlStream.ReadFully(Strea m stream, Byte[] buffer, Int32 offset, Int32 count)

at MySql.Data.MySqlClient.MySqlStream.LoadPacket()

at MySql.Data.MySqlClient.MySqlStream.ReadPacket()

at MySql.Data.MySqlClient.NativeDriver.GetResult(Int3 2& affectedRow, Int64& insertedId)

at MySql.Data.MySqlClient.Driver.GetResult(Int32 statementId, Int32& affectedRows, Int64& insertedId)

at MySql.Data.MySqlClient.Driver.NextResult(Int32 statementId, Boolean force)

at MySql.Data.MySqlClient.MySqlDataReader.NextResult( )

at MySql.Data.MySqlClient.MySqlCommand.ExecuteReader( CommandBehavior behavior)

--- End of inner exception stack trace ---

at MySql.Data.MySqlClient.ExceptionInterceptor.Throw( Exception exception)

at MySql.Data.MySqlClient.MySqlConnection.Throw(Excep tion ex)

at MySql.Data.MySqlClient.MySqlConnection.HandleTimeo utOrThreadAbort(Exception ex)

at MySql.Data.MySqlClient.MySqlCommand.ExecuteReader( CommandBehavior behavior)

at MySql.Data.MySqlClient.MySqlCommand.ExecuteReader( )

at PRoConEvents.CBattlelogCache.MySqlLookup(String name, RequestTypeEnum requestType)

what am i doing wrong ?

 

Edit : just deleted the whole db and created new one. working perfect now.

[ImportBot]

Originally Posted by MorpheusX(AUT)*:

 

I would guess that either:

1) Your MySQL-Server has had unexpected problems and went down

2) Your MySQL-Server was overloaded and thus failed to respond within the needed timespan

3) Either the server running your Procon layer or the one running your MySQL-client are having network troubles (overload, outage, DDoS, etc...)

 

You're probably not doing anything wrong. Did that occurr once or several times (also, just at a specific time or e.g. multiple times during the day)?

[ImportBot]

Originally Posted by XpKiller*:

 

@Morpheus

that are the same problems i had with my stats plugin as well. The defaults of the connector need to be adjusted. i guess the database got to slow or the index dont work.