Mule缓存-内存中的群集-反序列化问题

时间:2018-06-21 04:18:00

标签: caching mule hazelcast object-storage

我的Mule 3.9应用程序公开了一个休息终点。 通过运行时管理器对应用程序进行集群和本地管理

条件是:开始批处理过程的端点应该是单例,这意味着在整个集群上应该只运行1个进程。如果再次调用其他端点,则它将进入http状态409

在此用例中,我使用了具有以下配置的Mule Caching-Clustered-In Memory版本

import pyodbc
import os.path
import string
import re
import sys
import time
from datetime import datetime

# Function for Do you want to continue
def fun_continue():
    # If you want to continue
    yesno = raw_input('\nDo you want to continue(Y/N)?')
    if yesno == 'Y':
        fun_comparison()
    else:
        sys.exit()


def fun_comparison():

    # Getting Input Value's
    file1 = raw_input('Enter the file1 name with path:')
    file_extension_old = os.path.splitext(file1)[1]

    #Condition check for the File extension, if it's ACCESS DB then ask for the table name
    if (file_extension_old == ".accdb")  or (file_extension_old == ".mdb"):
        table_name_old = raw_input('Enter table name:')

    file2 = raw_input('Enter the latest file name:')
    file_extension_latest = os.path.splitext(file2)[1]

    #Condition check for the File extension, if it's ACCESS DB then ask for the table name
    if (file_extension_latest == ".accdb") or (file_extension_latest == ".mdb"):
        table_name_latest = raw_input('Enter table name:')

    file3 = raw_input('Give the file name to store the comparison result:')
    print('Files comparison is running! Please wait...')

    # Duration Calculation START TIME
    start_time = datetime.now()

    # Code for file Comparison
    try:
        #Condition check for the ACCESS FILE -- FILE 1
        if (file_extension_old == ".accdb") or (file_extension_old == ".mdb"):
            conn_string_old = r'DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};DBQ='+file1+';'
            con_old = pyodbc.connect(conn_string_old)
            cur_old = con_old.cursor()

            #Getting Column List
            res_old = cur_old.execute('SELECT * FROM '+table_name_old+' WHERE 1=0')
            column_list = [tuple(map(str, record_new))[0] for record_new in res_old.description]
            column_list = ';'.join(column_list)

            #For Getting Data
            SQLQuery_old = 'SELECT * FROM '+table_name_old+';'
            rows_old = cur_old.execute(SQLQuery_old).fetchall()
            records_old = [tuple(map(str,record_old)) for record_old in rows_old]
            records_old = [";".join(t) + "\n" for t in records_old]
            records_old = set(records_old)
            records_old = map(str.strip, records_old)
            #print records_old
        else:
            with open(file1) as a:
                column_list = a.readline()
                column_list = re.sub(r"[;,|^~]", ";", column_list)
                a = set(a)
                sete = map(str.strip, a)
                setf = [re.sub(r"[;,|^~]", ";", s) for s in sete]
                records_old = [";".join(map(str.strip, i.split(";"))) for i in setf]

        #Condition check for the ACCESS FILE -- FILE 2
        if (file_extension_latest == ".accdb") or (file_extension_latest == ".mdb"):
            conn_string_new = r'DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};DBQ='+file2+';'
            con_new = pyodbc.connect(conn_string_new)
            cur_new = con_new.cursor()

            #Getting Column List
            res_new = cur_new.execute('SELECT * FROM '+table_name_latest+' WHERE 1=0')
            column_list = [tuple(map(str, record_new))[0] for record_new in res_new.description]
            column_list = ';'.join(column_list)

            SQLQuery_new = 'SELECT * FROM '+table_name_latest+';'
            rows_new = cur_new.execute(SQLQuery_new).fetchall()
            records_new = [tuple(map(str,record_new)) for record_new in rows_new]
            records_new = [";".join(t) + "\n" for t in records_new]
            records_new = set(records_new)
            records_new = map(str.strip, records_new)
            #print records_new
        else:
            with open(file2) as b:
                column_list = b.readline()
                column_list = re.sub(r"[;,|^~]", ";", column_list)
                b = set(b)
                sete = map(str.strip, b)
                setf = [re.sub(r"[;,|^~]", ";", s) for s in sete]
                records_new = [";".join(map(str.strip, i.split(";"))) for i in setf]

        column_list = column_list.strip()
        column_list = column_list.replace('; ', ';').strip(' ')
        with open(file3, 'w') as result:
            result.write(column_list + '\n')
            for line in records_new:
                if line not in records_old:
                    result.write(line + '\n')

    except Exception as e:
        print('\n\nError! Files Comparison completed unsuccessfully.')
        print('\nError Details:')
        print(e)

    # Duration calculation END TIME
    end_time = datetime.now()
    print('Duration: {}'.format(end_time - start_time))
    # Calling Continue function
    fun_continue()


# Calling Comparison function
fun_comparison()
input()

我的流程如下所示-

<ee:object-store-caching-strategy name="caching_strategy" doc:name="caching_strategy" keyGenerationExpression="some_key" synchronized="false" entryTTL="14400000" persistent="false" />

如您所见,我将具有1个键值对的java.util.HashMap放入缓存中,并检查它是否已经存在

<flow name="some-flow" doc:description="some-flow">
   <message-properties-transformer scope="invocation" doc:name="Intialize Message Properties" mimeType="application/java">
        <add-message-property key="messageId" value="#[message.rootId]"/>
   </message-properties-transformer>

<ee:cache doc:name="inititiation" cachingStrategy-ref="caching_strategy" >
    <logger message="process cache miss" level="INFO" doc:name="process cache miss"/>
    <set-payload doc:name="initialize cache map" value="#[{'id' : flowVars.messageId}]" />
</ee:cache>

<choice doc:name="Is process already running ?" >
    <when expression="#[payload.id == flowVars.messageId]">
        <logger message="New process started" level="INFO" />
    </when>
    <otherwise>
        <logger message="Process is already running" level="WARN" />
    </otherwise>
</choice>
</flow>

实际功能在集群中非常有用,并且可以达到目的!

如何应用程序日志中充满了以下 <set-payload doc:name="initialize cache map" value="#[{'id' : flowVars.messageId}]" /> 条语句

**WARN**

我不确定是什么问题?缓存中的对象是可序列化的java.util.HashMap,只有键值对是String。

我感觉到一些类加载器问题,但无法解决这个问题。

有人有什么线索吗?

谢谢 维卡斯

1 个答案:

答案 0 :(得分:0)

动手实践并设法通过以下配置解决问题-

<ee:object-store-caching-strategy name="caching_strategy" doc:name="caching_strategy" keyGenerationExpression="some_key" synchronized="false" >

    <!-- this is because my flow eturns different message than cache"
    <ee:serializable-event-copy-strategy/>

    <!-- manged store without persistance -->
    <managed-store storeName="MyTaskInMemoryStoreForClusteredCaching" 
               maxEntries="1" entryTTL="14400000" 
               expirationInterval="3660000" persistent="false"/>


</ee:object-store-caching-strategy>